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EXECUTIVE  SUMMARY 


Research  Requirement: 

The  purpose  of  the  research  was  to  determine  the  components  of  the  constructs  associated 
with  a  Joint  and  Expeditionary  Mindset,  the  fundamental  attributes  embodied  therein,  the 
operational  capabilities  to  which  it  is  relevant,  and  the  environments  suitable  to  enhance  ground 
component  Expeditionary  Mindset  and  the  associated  Joint  capabilities.  The  specific  aim  was  to 
develop  a  computer-mediated  training  environment  to  prepare  ground  component  forces  with  the 
necessary  cognitive  skills  for  the  emerging  challenges  of  a  Joint  and  Expeditionary  force. 


Procedure: 

A  key  aspect  of  our  methodology  was  to  assemble  a  group  of  technical,  operational,  and 
programmatic  experts  to  examine  assumptions,  methodology,  and  conclusions  in  a  collaborative 
working  group  over  an  extended  period  of  time.  Participants  in  the  working  group  consisted  of 
the  investigators,  including  behavioral  scientists  and  recently  retired  Army  officers  whose 
primary  responsibility  at  the  time  of  the  investigation  was  identifying  and  promulgating  lessons 
learned  from  Iraq  and  Afghanistan  about  asymmetrical  warfare  to  active  duty  military  units. 

The  initial  responsibility  of  the  working  group  was  to  identify  capabilities  of  high  priority  in 
Joint  and  expeditionary  operations,  priorities  for  which  there  is  a  convergence  of  evidence  from 
different  sources.  The  Joint  Capabilities  Integration  and  Development  System  (JCIDS)  provided 
a  structured  methodology  for  identification  of  capability  gaps  and  solutions.  We  utilized  a 
general  strategy  typical  of  pre-systems  acquisition  to  increase  the  likelihood  that  the  innovation 
can  be  transferred  to  an  environment  of  use  within  the  Department  of  Defense  (DoD). 

We  utilized  a  spiral  development  approach  that  has  become  predominant  in  software 
development.  This  reality  demands  persistent  contact  with  end  users  and  stakeholders  to  be  able 
to  trace  the  evolution  implicit  in  the  uses  of  a  new  product  or  service,  to  adapt  what  one  comes  to 
understand  as  the  incidental  features  of  a  product,  to  come  to  a  deeper  understanding  of  the 
unique  and  essential  attributes  of  the  product,  and  to  ensure  that  modifications  and  upgrades 
preserve  the  unique  and  essential  attributes  over  the  life  cycle  of  the  product.  The  development 
team  maintained  a  habitual  relationship  with  end  users  in  the  Maneuver  Captains  Career  Course 
(MC3)  over  three  years  including  changes  in  the  chain  of  command  and  cadre. 

We  utilized  multiple  methodologies,  including  qualitative  inquiry,  because  this  is 
considered  a  best  practice  in  usability  engineering.  Crystallization  and  constant  comparison  were 
the  essential  analytical  tools.  Crystallization  involves  finding  fit  and  coherence  in  the  evidence 
from  users  about  the  utility  and  value  of  an  evolving  design.  It  helps  identify  unintended 
consequences  of  design  particular  features,  identify  subtle  functional  interactions  among 
features,  and  differentiate  the  essential  from  the  incidental.  Constant  comparison  involves 
crosschecking  among  the  interpretations  of  different  sources  of  evidence  and  vetting  the 
interpretations  with  end  users  from  whom  design-relevant  evidence  was  gathered.  Consistent 
with  best  practices  in  usability  engineering,  we  utilized  a  format  for  concisely  capturing 
requirements  implicit  in  the  experience  and  opinions  of  end  users  and  stakeholders.  Value  added 
by  this  practice  is  that  it  shows  informants  that  their  time  was  not  wasted;  it  shows  them  that  the 
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software  developers  were  listening  to  what  they  had  to  say;  and  it  reveals  the  potential  impact  of 
their  time  and  their  opinions  in  the  requirements  capture  process. 

The  final  stage  of  our  usability  engineering  process  was  initiated  once  the  design  of 
Socrates  Window  was  stabilized  through  formal  systems  engineering  change  control  and 
configuration  management.  Two  major  groups  of  experts  were  utilized  in  usability  trials  with 
Socrates  Window:  (a)  personnel  from  the  Army  Center  for  Enhanced  Performance  (ACEP)  who 
had  relatively  little  knowledge  of  Army  operations,  and  (b)  personnel  associated  with  the  Army 
Air  Defense  community  who  were  knowledgeable  about  Army  operations  but  from  a  different 
specialty  relative  to  the  MC3.  The  fundamental  query  of  these  outside  experts  was  whether  and 
to  what  extent  Socrates  Window  provides  outsiders  with  expeditious  visibility  into  the  MC3 
sufficient  to  reveal  their  opportunities  for  influence  on  the  MC3. 

Findings: 

Socrates  Window  provides  an  open-source  web-based  solution  that  has  the  potential  to 
facilitate  interactions  between  students,  instructors,  and  outside  experts  that  blends  classroom- 
based  learning  with  distance  learning. 

Evidence  indicates  general  interest  in  the  concept  among  stakeholders  and  end  users  such 
as  Small  Group  Instructors  (SGIs)  in  Army  training  and  education.  While  the  evidence  indicates 
that  Socrates  Window  has  both  value  and  utility,  it  is  not  currently  usable  in  the  MC3 
environment  primarily  because  of  IT  and  security  constraints  that  de-motivate  use  of  social 
networking  tools.  Evidence  from  demonstrations  and  interactions  with  experts  outside  the  MC3 
indicated  that  value  of  the  portal  would  be  increased  significantly  by  the  addition  of  a  portal  view 
that  guides  collaborative  work  and  further  development  of  a  common  scenario,  especially  to  the 
extent  that  the  common  scenario  pushes  subgroups  of  participants  to  and  slightly  beyond  the 
limits  of  their  experience. 

Utilization  and  Dissemination  of  Findings: 

Socrates  Window  will  have  value  to  instructors  whose  subject  matter  includes  content 
and  considerations  that  transcend  the  knowledge  and  experience  of  their  specialty.  To  be  useful 
within  the  Department  of  Defense,  the  innovation  will  have  to  be  implemented  by  organizations 
that  have  responsibility  for  IT  and  security  of  web-based  tools.  This  will  be  facilitated  by  the  fact 
that  Socrates  Window  will  not  be  protected  as  intellectual  property;  it  has  been  developed  as  an 
open-source  solution.  This  open-source  solution  will  be  promulgated  through  publications  that 
articulate  its  unique  and  essential  attributes.  More  specifically,  Socrates  Window  can  be  valuable 
in  ongoing  development  of  the  common  core  and  use  of  the  common  scenario  across  Captains 
Career  Courses  in  the  Army. 
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A  COMPUTER  MEDIATED  LEARNING  ENVIRONMENT  FOR  A  JOINT  AND 

EXPEDITIONARY  MINDSET 

CONCEPT  REFINEMENT 

Needs  Identification 

Joint  and  Expeditionary  Mindset  (JEM) 

When  considering  JEM,  it  is  clear  that  the  future  force  will  not  have  the  luxury  to  provide 
the  Soldier  with  extensive  training  for  particular  regional  assignments  and  cultures.  Instead, 
Soldiers  must  be  ready  to  be  assigned  anywhere  in  the  world  on  relatively  short  notice.  They 
must  be  prepared  to  adapt  rapidly  to  situations  in  which  factors  of  mission,  enemy,  terrain, 
troops,  time,  civilians  (METT-TC)  are  unfamiliar  or  changing  rapidly.  They  must  learn  to  learn. 

The  reflections  of  LTG  Petraeus  on  Iraq  are  instructive.  They  point  to  some  of  the  issues 
and  the  level  of  analysis  required  in  the  development  of  training  systems  for  cognitive  skills 
needed  in  ill-defined  cross-cultural  situations  (Petraeus,  2006): 

•  Understand  local  organizations  and  coordinate  with  specific  individuals  well  enough  to 
enable  locals  to  take  control  of  their  own  destiny. 

•  Make  as  many  of  the  locals  as  possible  feel  that  they  have  a  stake  in  stability  and 
reconstruction  operations  in  which  the  U.S.  military  personnel  are  significantly  involved. 

•  Understand  the  costs  and  benefits  of  every  tactical  decision  in  terms  of  its  possible 
unintended  consequences  on  the  attitudes  and  actions  of  the  local  population. 

•  Develop  networks  for  human  intelligence  at  the  level  of  neighborhoods  because  this  is  scale 
on  which  terrorists’  tactics  of  intimidation  and  disruption  are  most  effective. 

•  Anyone  and  everyone  can  become  involved  in  civil  affairs  during  stability  and  reconstruction 
operations  irrespective  of  training  in  this  area  prior  to  deployment. 

•  Understand  the  interconnections  among  local  institutions  which  provide  the  services  on 
which  a  local  population  depends. 

•  Understand  the  values  and  associated  cultural  practices  which  give  the  local  population  its 
sense  of  identity  and  self  worth. 

•  Understand  the  cultural  ecology  including  political,  economic,  social,  and  physical 
conditions  of  people’s  daily  lives. 

LTG  Petraeus  emphasizes  that  one  of  the  major  obligations  of  Commanders  to  junior 
leaders  is  “to  do  everything  possible  to  train  them  before  deployment  for  the  various  situations 
they  will  face,  particularly  for  the  most  challenging  and  ambiguous  ones”  (Petraeus,  2006,  p.  7). 

Such  demands  of  the  contemporary  operating  environment  (COE)  also  have  been  noted 
recently  by  other  military  scholars  who  are  mindful  of  lessons  learned  throughout  the  history  of 
counterinsurgency  (COIN;  e.g.,  Kilcullen,  2006;  Scales,  2006;  Yates,  2006).  The  most  important 
implications  are  to  understand  vulnerability  of  systems  and  infrastructure  on  which  social 
stability  depends.  The  other  is  the  importance  of  human  intelligence  because  it  is  both  a  robust 
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means  for  our  adversaries  to  identify  and  exploit  vulnerability  and  a  means  for  U.S.  forces  to 
detect  or  defeat  insurgents  who  employ  idiosyncratic  means  of  attack  (Meigs,  2003). 

As  implied  by  the  observations  of  Petraeus  (2006),  Kilcullen  (2006,  p.  1)  aptly  describes 
the  most  beneficial  activities  as  those  reminiscent  of  beat  cop  behaviors.  He  emphasizes  that 
individuals  should  develop  “a  framework  in  which  to  fit  every  new  piece  of  knowledge  you 
acquire.  Project  handover  notes  from  predecessors”  (Kilcullen,  2006,  p  2). 

Military  Training  and  Education 

The  objective  of  our  ongoing  research  and  development  (R&D)  reported  here  is  to 
determine  the  constructs  associated  with  JEM  and  to  develop  a  computer-mediated  learning 
environment  to  foster  that  mindset  in  ground  combat  personnel  (Riccio,  Lerario,  Cornell 
d’Echert,  Pound,  Brunye,  &  Diedrich,  2006). 

We  believe  that  a  developmental  perspective  must  be  at  the  heart  of  any  approach  that 
purports  to  address  or  change  mindset.  From  a  developmental  perspective,  early  to  mid-career 
Army  officers  appear  to  be  the  students  who  would  most  benefit  from  an  educational  innovation 
directed  at  a  JEM.  More  specifically,  the  Maneuver  Captains  Career  Course  (MC3)  is  an 
appropriate  curriculum  for  JEM  given  that  it  focuses  on  Combined  Arms  Warfighting  at  the 
tactical  level  across  the  full  spectrum  of  combat. 

The  MC3  divides  the  program  of  instruction  into  two  phases:  the  Company  Phase  and  the 
Brigade-Battalion  Phase.  We  worked  with  the  Company  Phase  in  which  all  officers  receive  the 
same  training  on  company-level  full  spectrum  operations.  There  are  several  modules  in  the 
Company  Phase,  each  approximately  one  week  in  duration.  Our  focus  is  on  the  module  that 
addresses  Stability  Operations  (SO). 

The  SO  module  expands  the  module  previously  entitled  the  Counterinsurgency  (COIN) 
module,  the  end-states  for  which  include:  (a)  quickly  analyze,  develop,  and  brief  a  tactically 
sound  course  of  action;  (b)  understand  the  enemy  (asymmetrical  threat  in  a  COIN  environment); 
(c)  understand  where  to  defeat  the  enemy  (terrain:  urban  environment);  (d)  understand  how  to 
defeat  the  enemy  in  the  course  of  limited  offensive  operations;  and  (e)  understand  the  capabilities 
of  the  assets  in  the  Stryker  Brigade  Combat  Teams  (SBCT)  Task  Organization. 

Pedagogical  Approach 

The  MC3  program  of  instruction  has  been  strongly  influenced  by  guidance  from  the  U.S. 
Army  Training  &  Doctrine  Command  (TRADOC)  and  Combined  Arms  Center  (CAC)  to 
emphasize  challenges  of  COIN  in  the  COE,  and  to  balance  offensive,  defensive,  and  stability 
operations.  The  classroom  environment  focuses  on  small  groups  of  up  to  16  Captains  led  by  a 
Small  Group  Instructor  (SGI).  Each  module  generally  utilizes  (a)  doctrinal  lectures,  (b)  historical 
vignettes,  (c)  readings  and  videos,  (d)  tactical  decision  exercises  (TDE)  in  which  students 
participate,  and  (e)  SGI-led  discussions.  SGI-led  TDEs  and  discussions  provide  opportunities  for 
collaborative  learning.  Instructors  can  provide  opportunities  for  collaborative  experiential 
learning  by  introducing  new  situations,  rich  in  detail,  to  a  class  along  with  general  principles  and 
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methods  for  making  meaning  of  the  details.  They  can  thus  reveal  to  students  the  ways  in  which 
knowledge  is  situated.  Understanding  the  influence  of  local  or  momentary  ecology  (e.g.,  physical 
and  cultural  surroundings)  on  an  unfolding  event  helps  students  generalize  from  their  own 
experiences  and  classroom  experiences  to  new  situations  (Brown,  Collins  &  Duguid,  1989; 
Gibson,  1991;  Lave  &  Wenger,  1991). 

The  common  details  and  scenarios  of  a  classroom  experience  also  foster  collaborative 
learning  among  the  students.  One  learner  can  make  personal  connections  between  sets  of 
concrete  details  that  initially  may  seem  bewildering  to  others.  In  a  collaborative  environment,  the 
idiosyncratic  connections  and  meanings  that  individual  students  make  of  the  classroom 
experiences  provide  opportunities  for  all  students  to  be  exposed  to  the  perspectives  of  others 
(Bandura,  1997;  Lewin,  1948).  This  provides  each  student  with  a  foundation  for  comprehending 
and  potentially  for  adopting  multiple  perspectives.  Accordingly,  a  key  objective  of  our  R&D  is 
to  help  achieve  and  implement  a  methodology  that  promotes  experiential,  participatory,  and 
collaborative  learning. 

The  Army  Guided  Experiential  Learning  (GEL)  model  (e.g.,  Clark,  2004;  Markley,  2007) 
holds  the  promise  of  meeting  many  educational  needs.  Nevertheless,  it  is  prudent  to  consider  a 
variety  of  alternative  methods,  including  problem-based  ones,  which  may  be  more  effective  for 
certain  kinds  of  learning  (e.g.,  see  Bransford,  Brown,  &  Cocking,  2000).  Problem-based  methods 
hold  the  promise  of  challenging  trainees  to  think,  and  to  discover  what  they  know  and  need  to 
know,  thereby  facilitating  deeper  understanding.  As  Bransford  et  al.  (2000)  note,  the  true 
challenge  is  to  identify  the  right  techniques  and  technologies  required  for  learners  to  achieve  the 
learning  objective  at  hand.  Whether  inquiry-based,  lecture-based,  technology-enhanced,  or  skills- 
based,  no  one  approach  will  be  effective  in  all  circumstances. 

The  argument  for  a  multifaceted  approach  underscores  the  general  need  for  a  learning 
environment  to  be  learner-centered,  knowledge-centered,  assessment-centered,  and  community- 
centered  (cf.,  Bransford  et  al.,  2000).  To  be  learner-centered,  a  learning  environment  must 
address  what  learners  bring  to  the  table-what  they  know,  what  they  don’t  know,  what  they  need 
to  learn,  and  what  they  are  motivated  to  learn.  To  be  knowledge-centered,  effective  learning 
environments  must  be  rooted  in  well-structured  bodies  of  knowledge  that  encourage  deep 
understanding  and  reflection.  They  encourage  sense-making  rather  than  mere  memorization.  To 
be  assessment-centered,  learning  environments  must  provide  formative  feedback  and  not  only 
summative  evaluation.  To  be  community-centered,  there  should  be  exploration  in  an  open 
environment  that  is  relevant  to,  and  reflects,  the  community  in  which  it  is  embedded.  Similarly, 
Wallace  (2006)  has  argued  that  the  Generative  Force  must  be  closely  coupled  to  the  Operational 
Force.  The  challenge  is  to  use  a  variety  of  technologies  and  techniques  that  find  the  “sweet  spot” 
at  the  intersection  of  these  objectives  (see  also,  Scales,  2006). 

In  the  following  sections,  we  describe  methods  and  progress  from  our  efforts  to  identify 
requirements  for  technological  support  and  educational  approaches  to  close  the  gap  between  the 
classroom  and  the  operational  environment. 


3 


Method 


Qualitative  Inquiry 

A  key  aspect  of  our  methodology  was  to  assemble  a  group  of  technical,  operational,  and 
programmatic  experts  to  examine  assumptions,  methodology,  and  conclusions  in  a  collaborative 
working  group  over  an  extended  period  of  time  (Riccio  et  ah,  2006;  Riccio,  Sullivan,  Klein, 
Salter,  &  Kinnison,  2004).  Participants  in  the  working  group  consisted  of  the  investigators, 
including  behavioral  scientists  and  four  recently  retired  Army  officers  whose  primary 
responsibility  at  the  time  of  the  investigation  was  identifying  and  promulgating  lessons  learned 
from  Iraq  and  Afghanistan  about  asymmetrical  warfare  with  active  duty  military  units. 

The  activities  of  the  standing  working  group  occurred  over  a  six  month  period  and  included 
participant  observation,  interviews,  document  analysis,  memo  writing,  constant  comparison, 
cross  checking,  achieving  a  balance  and  tension  of  multiple  perspectives,  and  development  of 
grounded  theory  (see  Camic  et  ah,  2003;  Denzin  &  Lincoln,  2003).  The  initial  responsibility  of 
the  working  group  was  to  identify  capabilities  of  high  priority  in  Joint  and  expeditionary 
operations,  priorities  for  which  there  is  a  convergence  of  evidence  from  different  sources.  The 
Joint  Capabilities  Integration  and  Development  System  (JCIDS)  provided  a  structured 
methodology  for  identification  of  capability  gaps  and  solutions  (Chainnan  of  the  Joint  Chiefs  of 
Staff,  2004).  Key  elements  of  the  JCIDS  methodology  include  Functional  Area  Analysis  (FAA), 
Functional  Needs  Analysis  (FNA),  Functional  Solution  Analysis  (FSA),  and  Post  Independent 
Analysis  (PI A). 

Three  focus  group  events  were  conducted  with  the  participants  of  our  standing  JCIDS-based 
working  group.  They  occurred  during  the  first  two  months  of  the  project  and  were  separated  by 
at  least  two  weeks.  Each  focus  group  event  occurred  over  a  two-day  period.  Sessions  on 
consecutive  days  were  between  two  and  four  hours  in  duration.  The  first  focus  group  identified 
facets  of  the  problem,  participants,  and  stakeholders.  The  second  focus  group  event  addressed  the 
FAA  and  FNA  and  the  third  addressed  the  FNA  and  the  FSA.  Considerations,  recommendations, 
and  opinions  of  the  working  group  were  documented  in  text  and  graphics.  This  helped  to  avoid  a 
narrow  focus  from  a  dominant  perspective.  In  the  spirit  of  the  PIA  step  in  the  JCIDS  process, 
conclusions  from  the  FNA  and  FSA  were  vetted  through  frequent  coordination  with  potential 
users  and  stakeholders.  The  intent  was  to  facilitate  development  of  capabilities  that  can  be 
transitioned  almost  immediately.  The  primary  source  of  stakeholders  and  users  was  the  MC3. 

After  the  FSA,  work  was  initiated  on  a  prototype  web-based  tool  that  can  be  used  in 
environments  such  as  the  MC3  to  help  SGIs  prepare  ground  component  forces  with  the  necessary 
cognitive  skills  for  the  emerging  challenges  of  a  Joint  and  Expeditionary  force.  Development  of 
the  tool  generated  additional,  more  refined,  questions  that  required  a  second  round  of 
collaborative  inquiry  with  subject  matter  experts  (SMEs).  This  iterative  cross-fertilization 
between  software  development  and  interactions  with  users  about  the  resulting  capabilities 
(“spiral  development”)  continued  throughout  the  project. 
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Surveys  and  Associated  Interviews 


Following  the  initial  qualitative  inquiry,  a  two-page  survey  was  distributed  to  52  current 
and  retired  Army  personnel  representing  a  variety  of  ranks  (COL  to  2LT)  and  years  of  military 
service  (1  to  3 1  years,  M=10  years)  to  reveal  additional  facets  of  the  problem  that  had  not  been 
addressed  in  the  FAA,  FNA,  and  FSA.  More  specifically,  the  survey  was  divided  into  two 
categories  of  questions.  The  first  set  of  questions  addressed  COE  knowledge  such  as  how 
important  information  about  the  COE  is  to  a  Soldier’s  survivability,  how  long  does  it  take  to 
gather  infonnation,  and  what  are  the  key  sources  of  information  gathered.  The  second  set  of 
questions  addressed  electronic  collaborative  tools  such  as  whether  such  tools  can  be  an  effective 
way  to  share  COE  information,  whether  people  are  willing  to  use  them,  whether  people  are  able 
to  use  them. 

There  were  two  types  of  survey  responses.  First,  a  5 -point  Likert-type  scale  was  used  to 
assess  participants’  level  of  agreement  with  certain  statements,  such  as  “Information  about  the 
COE  is  important  to  a  Soldier’s  survivability.”  Second,  participants  could  rank  a  given  list  of 
items,  such  as  ranking  sources  of  COE  information. 

Broader  Interviews 

To  facilitate  analysis,  follow-up  interviews  with  two  military  education  experts  were  then 
conducted  to  help  interpret  the  survey  results  and  to  stimulate  broader  discussion  with  the  two 
experts  (Tashakkori  &  Teddlie,  1998).  In  the  second  part  of  each  interview,  the  two  experts  were 
asked  for  their  opinions  on  issues  related  to  learner-centered,  knowledge-centered,  assessment- 
centered,  and  community-centered  aspects  of  military  training  and  education  that  were  most 
important  in  the  context  of  the  findings  from  the  FAA,  FNA,  and  FSA. 

Results  and  Discussion 


Functional  Area  Analysis 

In  the  JCIDS-based  FAA,  the  working  group  identified  the  implications  of  Joint 
Operations  Concepts  (JOpsC),  Joint  Operating  Concepts  (JOC),  and  Joint  Functional  Concepts 
(JFC)  for  training  JEM.  The  relevant  JOpsC  are  fully  integrated,  expeditionary,  networked, 
decentralized,  adaptable,  and  decision  superior  (DoD,  2003).  An  important  implication  of  these 
attributes  is  that  individuals  and  units  must  be  rapidly  deployable,  employable  and  sustainable  in 
areas  of  operation  that  may  have  a  minimum  of  existing  infrastructure. 

The  JOpsC  are  fostered  by  timely  and  accurate  communication  about  relevant  situations 
and  lessons  learned.  Relevance,  timeliness,  and  accuracy  are  fostered  by  two-way 
communication  that  helps  individuals  understand  the  contexts  within  which  information  is 
presented  and  received,  elaborate  on  actual  or  potential  misunderstandings,  and  establish  the 
credibility  of  the  source.  There  is  rapid  growth  in  capabilities  that  help  link  individuals  who  have 
a  need  to  know  with  appropriate  experts. 
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The  JOC  span  Full  Spectrum  Operations  (DoD,  2004b).  The  biggest  challenges  for  Joint 
and  expeditionary  operations  are  rapid  transitions  between  offensive,  defensive,  and  stability 
operations.  There  are  profound  issues  in  these  transitions  for  individuals  and  for  units  over  short 
and  long  time  scales.  Rapid  transitions  that  are  especially  difficult  involve  in-stride  changes 
between  lethal  and  nonlethal  actions.  With  respect  to  longer-term  issues,  the  working  group 
identified  the  following  question  as  a  powerful  guideline  for  training:  “What  would  Soldiers  do 
differently  if  they  knew  they  had  to  stay  and  light  for  the  duration  of  the  operation?”  This 
mindset  helps  a  Soldier  take  ownership  of  the  job  of  the  person  to  be  replaced  rather  than  merely 
taking  the  place  of  that  person. 

In  the  context  of  the  JOpsC  attribute  of  decentralized,  and  especially  in  the  context  of  the 
stability  operations,  the  Joint  Functional  Concept  (JFC)  of  Joint  Command  &  Control 
emphasizes  the  importance  of  a  subordinate  leader ’s  ability  to  act  on  the  basis  of  broad 
statements  of  commander’s  intent  (DoD,  2004a).  Similarly,  it  is  important  for  commanders  to  be 
able  to  provide  subordinates  with  clear  direction  without  presumptive  detail  about 
implementation.  In  essence,  commanders  and  subordinates  must  learn  to  strike  a  balance  in  the 
natural  tension  between  command  and  control. 

Functional  Needs  Analysis 

In  the  JCIDS-based  FNA,  the  working  group  focused  on  two  major  components  of 
expeditionary  mindset:  Getting  there  and  being  there.  “Getting  there”  skills  and  issues  include: 

(a)  no-notice  or  short-notice  deployment  sequence;  (b)  area  studies  to  gain  general  knowledge  of 
potential  threats  and  environments;  and  (c)  basic  Soldier  skills  such  as  move,  shoot, 
communicate,  and  first  aid.  “Being  there”  skills  and  issues  include  (a)  coordination  and 
interaction  with  other  government  agencies  (OGA)  and  non-government  organizations  (NGO); 

(b)  population  engagement,  both  the  leadership  and  masses;  (c)  offensive  action  to  destroy 
insurgent  elements;  and  (d)  intelligence  collection  and  analysis.  No  matter  where  “there”  is,  the 
problem  sets  and  resources  will  be  similar:  tribal  and  political,  OGA  and  NGO  interaction, 
offensive  and  defensive  operations. 

The  intent  of  the  working  group  was  to  identify  urgent  needs  related  to  JEM  in  current 
operating  environments.  Given  this,  and  in  retrospect,  it  is  not  surprising  that  we  identified  some 
nascent  solutions  in  the  activities  of  networked,  decentralized,  and  adaptable  forces  (both  the 
Generative  and  Operation  Force).  Two  kinds  of  emerging  solutions  are  represented  by  the  Battle 
Command  Knowledge  System  (BCKS)  and  CAVNET.  The  CAC  BCKS  is  an  online  knowledge 
management  system  for  the  generation,  storage  and  rapid  retrieval  of  information  (Kerr,  2006). 
BCKS  includes  a  forum  for  interaction  between  individuals  as  well  as  a  database  of  documents 
(Warrior  Knowledge  Base).  The  database  includes  many  thousands  of  knowledge  objects.  There 
are  capability  gaps  in  BCKS,  however,  that  derive  from  the  quantity  of  relevant  material  and 
number  of  individuals  to  which  it  provides  access. 

One  capability  gap  is  the  difficulty  in  finding  the  right  information  in  the  right  amount 
and  in  the  time  the  user  has  available.  This  is  a  common  problem  with  knowledge  databases  that 
provide  a  plethora  of  information  that  is  apparently  or  actually  relevant  to  a  user’s  needs.  To 
address  this  gap,  methods  of  training  are  being  considered  which  could  help  achieve  a  virtual 
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“right  seat  ride”  for  novice  users  (Kerr,  2006).  We  believe  that  this  is  the  most  promising 
approach  to  making  a  database  both  usable  and  useful. 

Another  capability  gap  is  in  measures  of  effectiveness  (MOE).  The  common  approach  to 
MOEs  in  a  knowledge  database  or  other  web-based  capabilities  is  to  count  number  of  objects  on 
the  site  or  how  many  times  the  site  is  accessed  (“hits”).  This  is  very  limiting,  and  it  sheds  little  or 
no  light  on  the  value  of  a  web  site  to  users.  MOE  are  needed  which  provide  insight  into  the 
nature  and  extent  of  the  impact  of  a  web  site  on  users,  whether  any  learning  has  taken  place,  or 
whether  the  meaning  or  use  made  of  the  infonnation  is  valid.  The  expert  guidance  implied  by  the 
virtual  right  seat  ride  could  provide  a  path  to  more  meaningful  MOE.  In  principle,  experts  could 
provide  subjective  assessments  of  individuals  they  are  guiding. 

CAVNET  is  a  method  of  sharing  information,  peer  to  peer,  which  conceptually  builds  on 
predecessors  such  as  Platoonleader.com  and  Companycommander.com.  CAVNET  has 
demonstrated  the  concept  for  providing  peer-to-peer  information  sharing  on  a  secure  network  in 
theater.  It  was  first  established  for  the  1st  Cavalry  Division  in  Iraq  in  April  2004  (PBS  Frontline, 
2005).  Since  then,  3rd  Infantry  Division  (“Marnenef ’)  and  the  4th  Infantry  Division 
(“Ironhorsenef ’)  have,  in  turn,  adopted  the  system  and  are  using  it  in  Iraq.  The  Internet  allows 
many  people  to  communicate  instantly  about  topics  of  mutual  interest.  There  is  demand  for  any 
such  resource  because  of  the  need  to  share  the  emerging  enemy  and  friendly  tactics,  techniques 
and  procedures.  There  is  a  need  to  achieve  competitive  advantage  against  a  networked, 
adaptable,  and  cellular  enemy  force.  In  essence,  CAVNET  demonstrates  one  method  to  provide  a 
trusted  source  of  knowledge  for  junior  leaders  on  a  time  scale  that  allows  these  users  to  establish 
credibility,  validity,  and  relevance  of  the  information  and  to  act  on  it,  that  is,  to  expedite  the 
sharing  of  actionable  intelligence. 

The  next  logical  step  is  to  utilize  such  a  capability  in  closing  the  gap  between  the 
institutional  Army  and  the  operational  Army — to  reduce  the  “. .  .lag  between  what  is  being  taught 
in  the  classrooms  and  evaluated  at  the  training  centers  and  what  is  being  executed  on  the  ground 
in  combat”  (PBS  Frontline,  2005).  Like  BCKS,  however,  the  success  of  CAVNET  brings 
capability  gaps  that  derive  from  potential  access  to  vast  amounts  of  information  and  large 
numbers  of  individuals. 

Functional  Solution  Analysis 

The  key  finding  from  our  JCIDS-based  working  group  was  that  there  is  a  need  for  a 
tighter  coupling  between  educational  institutions  and  the  current  operations  in  theater.  Given  that 
the  MC3  arguably  is  on  the  cutting  edge  in  closing  this  gap,  the  question  becomes  how  best  to 
establish  even  closer  integration  of  the  educational  and  operational  environments.  The  JCIDS- 
based  FSA  converged  on  computer-based  tools  that  allow  experts  in  the  COE  (ECOE)  to 
communicate  directly  with  students  and  become  a  resource  to  SGIs  in  the  MC3.  The  toolset  will 
allow  SGIs,  students,  and  ECOEs  to  interact  online  regarding  key  issues,  topics,  or  documents  of 
interest  (Figure  1). 
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Figure  1.  Opportunities  for  COE  experts  to  assist  SGIs  in  closing  the  gap  between 
classroom  and  COE. 

By  enabling  such  interaction,  students  can  explore  how  concepts  learned  in  the  classroom 
relate  to  current  issues.  They  can  actively  pull  relevant  infonnation  from  experts  in  theater  or 
who  are  otherwise  closer  to  the  roles  in  which  students  will  find  themselves  after  deployment. 
The  key  insight  is  that  a  dialogue  must  develop  that  allows  exploration  through  questions  and 
answers  in  which  the  student  can  be  guided  to  make  non-trivial  links  between  doctrine, 
classroom  exercises,  and  current  practice.  The  notion  is  that,  by  guiding  student  dialogue  with 
ECOEs,  instructors  can  facilitate  early  rudimentary  “Right  Seat  Rides”  that  begin  familiarization 
and  make  abstract  lessons  more  concrete  and  actionable.  The  outcome  should  be  Soldiers  who 
have  learned  to  learn,  and  who  are  ready  to  learn  once  deployed  into  a  changing  or  ill-defined 
situation. 

Post  Independent  Analysis 

The  primary  thrust  of  work  is  not  the  software  tool  to  support  dialog  -  indeed,  simple 
functions  like  chat  and  virtual  whiteboards  already  exist.  Our  focus  is  in  how  to  structure  such  an 
environment  to  achieve  a  significant  improvement  in  learning,  and  in  particular,  its  ultimate 
impact  on  cognition  and  action  in  the  COE.  Our  interactions  with  users  and  stakeholders  were 
critical  in  the  development  and  integration  of  the  materiel  and  nonmaterial  components  of  the 
capability.  Concepts  about  the  software  tool,  the  needs  for  it,  and  the  use  of  it  were  examined 
collaboratively  as  the  concepts  developed  and  as  they  were  instantiated  in  initial  builds. 
Interactions  with  users  and  stakeholders  utilized  a  multifaceted  methodology  including 
interviews,  document  analysis,  and  naturalistic  observations  (Camic  et  ah,  2003;  Denzin  & 
Lincoln,  2003;  Tashakkori  &  Teddlie,  1998).  The  concurrent  development  of  a  grounded  theory 
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builds  on  the  framework  of  Bransford,  et  al.  (2000).  The  reciprocal  influence  among  these 
concurrent  activities  instantiated  a  spiral  development  approach  consistent  with  an  evolutionary 
acquisition  strategy  (Defense  Acquisitions  University  [DAU],  2003).  Our  assumption  is  that  this 
framework  for  our  R&D  would  facilitate  transition  of  our  innovation  (see  section  on  “Critical 
Considerations  for  Transition”). 

Survey  and  Associated  Interview  Results 

COE  Knowledge.  Respondents  believed  that  information  about  the  COE  is  very 
important  (M  =  4.65/5,  SD  =  0.59)  to  a  Soldier’s  survivability.  When  asked  to  estimate  how 
long  it  takes  a  Soldier  to  acquire  critical  information  about  the  COE,  50%  of  participants 
responded  1-2  months,  while  another  20%  indicated  2-4  months.  Surprisingly,  approximately 
20%  indicated  that  it  takes  only  1-2  weeks  to  acquire  information  on  the  COE.  Results  from 
follow-up  interviews  revealed  that  these  respondents  may  fall  into  two  very  different  categories: 
members  of  rapid  deployment  teams  and  novices.  The  former  are  trained  to  gather  and  share 
information  on  the  COE  in  a  rapid  manner,  while  the  latter  may  simply  be  unaware  of  all  the 
information  they  need  to  know.  The  remaining  10%  of  the  participants  chose  either  1-2  days  or 
4-6  months. 

Soldiers  indicated  that  they  most  commonly  use  first  hand  experience  and  right  seat  rides 
(RSRs)  as  sources  of  critical  infonnation  about  the  COE,  followed  in  decreasing  order  by  verbal 
word-of-mouth  (from  lateral  ranks),  verbal  word-of-mouth  (from  superiors),  verbal  word-of- 
mouth  (from  subordinates),  unofficial  written  or  electronic  documents,  and  official  written  or 
electronic  documents.  RSRs  can  be  particularly  valuable  in  learning  the  geography  of  the  COE. 
However,  several  respondents  noted  that  the  quality  of  RSRs  can  vary  drastically.  Interviewees 
indicated  that  the  attitude  of  the  incoming  or  outgoing  units  can  greatly  impact  the  quality  of  a 
RSR.  For  example,  if  the  outgoing  unit  is  focused  on  going  home  after  a  lengthy  deployment, 
then  they  may  not  be  motivated  to  conduct  a  thorough  RSR.  Similarly,  if  the  incoming  unit 
believes  that  it  knows  everything  about  the  COE  already,  they  may  not  attend  to  the  information 
in  the  RSR  as  closely  as  perhaps  they  should. 

Soldiers  ranked  both  official  and  unofficial  written  or  electronic  documents  as  being  the 
two  least  commonly  used  sources  of  COE  information.  Interviewees  indicated  a  variety  of 
reasons  why  these  potentially  valuable  sources  of  information  are  not  commonly  used.  For 
instance,  Soldiers  may  be  required  to  read  through  often  voluminous  amounts  of  data.  While 
units  often  do  make  storyboards,  briefings,  and  other  materials  available  to  other  units,  it  would 
be  time  consuming  to  review  it  all.  Furthermore,  Soldiers  may  be  unable  to  review  such 
materials  because  of  limited  SIPRNET  access. 

Not  surprisingly,  the  majority  of  respondents  (55%)  stated  that  some  COE  information 
sources  are  underutilized.  However,  45%  of  respondents  believed  that  there  are  no  underutilized 
sources.  Interviewees  were  surprised  by  the  number  of  respondents  who  answered  this  way. 
They  suggested  that  less  experienced  Soldiers  might  not  be  aware  of  what  other  types  of 
information  could  help  them  to  learn  about  the  COE.  Indeed,  ‘yes’  responses  were  positively 
associated  with  years  of  military  service. 
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Electronic  Collaborative  Tools .  Respondents  believed  that  electronic  collaborative  tools 
are  useful  for  sharing  COE-relevant  information  (M  =  4. 1 1/5,  SD  =  0.81)  and  that  they  would  be 
likely  to  use  tools  for  distance  learning  (M  =  3.98/5,  SD  =  0.77).  They  ranked  the  following 
communication  modalities  according  to  their  likelihood  of  use  in  decreasing  order: 

Asynchronous  one-to-one  communications  (such  as  an  electronic  mail  service);  Synchronous 
one-to-one  communications  (such  as  a  text-based  real-time  messaging  service  between  two 
individuals);  Asynchronous  one-to-many  communications  (such  as  posting  on  an  electronic 
bulletin  board);  Synchronous  one-to-many  communications  (such  as  a  text-based  real-time 
messaging  service  between  multiple  individuals). 

Respondents  also  indicated  they  would  be  both  willing  and  able  (M  =  3.86/5,  SD  =  0.89) 
to  participate  in  teaching  about  the  COE  using  electronic  collaborative  tools.  Furthermore, 
Soldiers  believed  that  electronic  collaborative  tools  for  distance  or  local  learning  can  contribute 
to  Soldier  survivability  by  bridging  the  gap  between  a  learning  environment  and  the  COE  (M  = 
4.06/5,  SD  =  0.75). 

Interviewees  were  somewhat  skeptical  of  the  respondents’  actual  ability  to  participate  in 
teaching.  Once  deployed,  Soldiers  have  many  demands  of  their  time.  Even  the  most  well- 
intentioned  Soldiers  may  find  that  they  either  do  not  have  the  time  to  teach  or  that  they  have 
unreliable  computer  access.  However,  the  key  to  promoting  usage  among  deployed  Soldiers  is  to 
make  it  as  easy  as  possible  for  them  to  use  the  tool.  Interviewees  recommended  strategies  such 
as  having  the  tool  be  compatible  with  Microsoft  Outlook  so  that  scheduled  sessions  would  be 
imported  into  Outlook.  Additionally,  interviewees  recommended  including  an  email 
functionality  through  which  users  can  send  links  to  other  users.  That  way  Soldiers  would  not  be 
required  to  log  in  to  the  tool  every  day  to  search  for  relevant  information  but  rather  the 
information  would  be  pushed  to  them. 

Broader  Interview  Results 

Community-centered  learning  environment.  Bransford  et  al.  (2000)  note  that 
community-centered  learning  environments  focus  on  two  key  elements:  (a)  an  environment  in 
the  classroom  that  encourages  open  discussion,  including  discussion  of  misconceptions,  in  a  non- 
judgmental  manner;  and  (b)  creating  an  environment  that  is  tied  in  tangible  ways  to  the 
community  it  serves  such  that  the  learning  is  seen  as  relevant.  From  this  perspective,  the 
following  key  insights  emerged  from  working  group  discussions,  surveys,  interviews,  and 
associated  document  reviews. 

The  first  key  element  of  community-centered  learning  environments  is  consistent  with 
the  Army’s  approach  to  After-Action  Reviews  (AAR)  and  belief  about  their  general  importance. 
AAR-style  interactions  are  common  in  the  MC3  and  in  other  venues  of  Anny  training  and 
education,  and  are  typically  conducted  in  an  open  and  non-judgmental  manner.  A  computer- 
based  tool  for  instructional  dialog  can  facilitate  AARs  by  reminding  participants  to  conduct  them 
in  a  nontrivial,  open  fashion. 

The  second  key  element  of  community  centered  learning  is  actually  the  central  focus  of 
the  current  work — helping  to  close  the  gap  between  the  classroom  and  the  COE.  Our  vision  is  for 
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students  and  SGIs  to  have  the  capability  for  web-based  interaction  with  individual  ECOEs 
almost  anywhere  at  anytime.  Any  progress  toward  achieving  this  vision  would  be  welcome  at  the 
MC3  and  presumably  many  other  sites  of  military  training  and  education. 

Bransford  et  al.  (2000)  describe  community-centered  learning  as  the  proper  backdrop  for 
learner-centered,  knowledge-centered,  and  assessment-centered  aspects  of  effective  educational 
environment.  Accordingly,  additional  issues  deriving  from  a  community-centered  approach  are 
described  below  in  the  context  of  the  implications  for  learner-centered,  knowledge-centered,  and 
assessment-c  entered  environments . 

Learner-centered  learning  environment.  Following  Bransford  et  al.  (2000),  we  assume 
that  the  key  element  of  being  learner-centered  is  to  ensure  that  the  learning  environment 
adequately  addresses  what  students  know,  do  not  know,  and  how  they  are  motivated  to  learn. 
Learners  start  at  different  places,  with  different  strengths  and  weaknesses,  and  different  gaps  in 
their  understanding.  From  this  perspective,  the  following  key  points  emerged  during  our 
investigation. 

Dialog  has  the  potential  to  help  reveal  what  students  know,  don’t  know,  and  how  they 
come  to  know.  The  small  group  environment  of  the  MC3  provides  opportunities  for  instructors 
to  dialog  with  students  and  for  students  to  dialog  with  each  other.  It  would  be  beneficial  to 
increase  opportunities  such  as  these  and  to  make  them  more  effective.  Two  ways  to  increase 
effectiveness  are  (a)  to  make  opportunities  for  dialog  available  when  students  are  most  highly 
motivated  to  receive  and  pull  information  from  others,  and  (b)  to  identify  and  promulgate  lessons 
learned  about  practices  in  instructional  dialog  that  students  and  instructors  find  to  be  useful. 

Computer-mediated  methods  of  dialog,  in  principle,  can  facilitate  the  capture  and 
archiving  of  best  practices  as  well  as  collaborative  insights  that  emerge  in  dialog  between 
novices  and  experts.  In  the  application  at  hand,  we  assume  that  best  practices  will  have  the 
characteristics  of  Socratic  dialog.  Instructors  or  other  experts  should  lead  students  to  discover 
connections  or  implications  for  themselves  rather  than  simply  telling  them  “the  answer.”  That  is, 
students  should  be  guided  in  how  to  think,  not  told  what  to  think. 

Computer-mediated  activities  also  allow  for  personal  structuring  of  information  that 
emerges  in  the  activities.  This  presents  tradeoffs  for  design  and  use  of  the  tool,  however,  because 
student-driven  structure  can  undennine  pedagogically-driven  structure  for  course  materials  and 
methodology.  Nevertheless,  a  capability  for  student-driven  structure  should  be  explored  given 
the  motivational  value  and  carry-over  effects  of  anything  that  promotes  active  learning. 

Web-based  dialog,  in  principle,  increases  access  to  a  wide  variety  of  experts.  In  the 
present  application,  the  most  valuable  experts  are  active  duty  personnel  in  theater  or  who  are 
otherwise  close  to  the  roles  in  which  students  will  find  themselves  after  deployment.  The  role  of 
such  ECOEs  in  web-based  dialog  is  important  to  the  extent  that  students  will  be  more  likely  to  be 
influenced  by  the  behavior  of  others  if  it  results  in  outcomes  they  value,  if  the  other  person  is 
similar  to  the  student  and  has  admired  status,  and  if  the  behavior  has  functional  value  (Bandura, 
1997). 
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Knowledge-centered  learning  environments.  To  be  knowledge-centered,  a  learning 
environment  must  go  beyond  simple  memorization  of  facts  and  disconnected  elements.  Rather, 
learning  environments  should  foster  sense-making-deep  understanding  involving  rich,  deep 
causal  connections  (Bransford  et  ah,  2000).  While  this  may  seem  obvious,  even  the  best 
curricula  can  benefit  from  continual  vigilance  about  the  coherence  and  relevance  of  the  big 
picture.  The  world  changes,  thus  the  key  integrating  themes  for  a  curriculum  may  need  to  be 
updated  from  time  to  time.  From  this  perspective,  the  following  key  insights  emerged  from  our 
investigation. 

Socratic  dialog  with  ECOEs  can  reveal  to  students  the  ways  in  which  knowledge  about 
the  operational  environment  is  situated  in  contemporary  nuances  of  political,  military,  economic, 
and  social  factors  as  well  as  infrastructure,  information,  physical  environment  and  time. 
Understanding  the  influence  of  local  or  momentary  ecology  on  an  unfolding  event  helps  students 
generalize  from  their  own  experiences  and  “classroom”  experiences  to  new  situations  (Gibson, 
1991;  Gibson,  1977;  Lave  &  Wenger,  1991). 

While  dialog  with  ECOEs  can  help  to  develop  integrated  sense-making — in  light  of 
connections  to  real  problems  in  the  COE — facts  and  issues  may  still  be  somewhat  difficult  to 
comprehend  in  absence  of  direct  experience  with  the  particular  roles  (e.g.,  of  Company 
Commanders)  in  the  COE.  Stories  by  ECOEs  and  instructors  can  help  provide  ways  to  integrate 
knowledge  into  meaningful  wholes.  Dialog  and  sharing  of  stories,  however,  must  be  grounded  in 
curriculum  objectives  and  associated  course  materials. 

One  important  element  of  curriculum,  as  a  whole,  is  the  developmental  perspective. 
Students  must  be  prepared  for  learning  at  the  level  required  in  any  particular  course.  Course 
content  and  methodology  takes  into  account  such  prerequisites.  This  is  difficult  to  ensure  in  a 
rapidly  changing  curriculum.  Such  situations  demand  extra  vigilance  by  instructors  in  attending 
to  the  meaning  that  students  make  or  are  able  to  make  of  course  material  and  experiences  in  the 
learning  environment.  In  this  respect,  web-based  dialog  with  ECOEs  is  not  likely  to  be 
pedagogically  effective  unless  ECOEs  can  be  resources  that  can  be  harnessed  by  the  SGIs. 

An  enabling  objective  of  our  approach  is  to  combine  the  best  of  knowledge  databases 
with  the  best  of  on-line  help  from  an  expert  by  providing  a  way  in  which  each  can  leverage  the 
strengths  of  the  other.  An  expert  can  help  a  novice  navigate  quickly  and  efficiently  through  an 
otherwise  potentially  overwhelming  amount  of  information  (e.g.,  course  materials).  At  the  same 
time,  the  need  to  know  about  something  specific,  something  which  is  potentially  available  in  a 
database,  grounds  and  contains  a  dialog  which  otherwise  could  become  tangential,  divergent,  or 
inefficient. 

Guidance  through  an  overwhelming  and  unfamiliar  body  of  information  is  no  different 
from  what  a  skilled  teacher  or  mentor  does  in  introductory  phases  of  any  curriculum.  Similar 
benefits  can  accrue  from  a  capability  that  would  enable  experts  to  guide  novices  “on  line”  as  a 
mechanism  of  distance  learning.  Grounding  dialog  between  a  student  and  an  instructor  in 
specific  learning  objectives  is  no  different  from  what  occurs  in  any  educational  situation.  The 
implication  in  the  present  case  is  to  provide  a  capability  for  on-line  dialog  between  a  third-party 
expert  and  a  student  to  be  harnessed  by  learning  objectives  of  the  instructor.  In  essence,  the 
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strategy  is  to  blend  classroom  education  with  distance  learning  in  a  way  that  each  facilitates  and 
gives  meaning  to  the  other. 

Assessment-centered  learning  environment.  Following  Bransford  et  al.  (2000),  learning 
environments  should  be  go  beyond  summative  assessments;  that  is,  identifying  what  students  can 
produce  on  a  test  or  a  paper  at  the  end  of  the  course.  While  meeting  standards  is  critical, 
Bransford  argues  that  assessment-centered  environments  must  focus  on  formative  assessment; 
that  is,  uncovering  misconceptions  and  providing  feedback  in  an  open  atmosphere.  From  this 
perspective,  the  following  points  emerged  from  working  group  discussions,  surveys,  interviews, 
and  associated  document  reviews. 

Many  current  methods  of  assessment  are  not  sufficient  in  that  they  do  not  peel  back  the 
layers  of  the  learning  process.  In  this  sense,  it  is  critical  to  develop  and  utilize  measures  of 
process  (MOP)  as  well  as  outcomes  or  measures  of  effects  (MOE).  Computer-mediated  activities 
can  provide  a  window  into  the  thinking  of  the  student.  In  principle,  one  can  trace  the  dialog  that 
leads  to  an  insight  or  to  a  dead  end.  The  selection,  use  and  storage  of  infonnation  can  be  traced. 
Such  sources  of  potential  MOPs  should  be  explored  in  the  development  of  the  tool. 

MOPs  will  be  valuable  for  continual  improvement  of  the  utilization  of  ECOEs  as  well  as 
in  the  improvement  of  learning  by  students.  The  contributions  of  ECOEs  can  be  assessed  with 
respect  to  the  ideals  of  Socratic  dialog.  SGIs  can  provide  such  feedback  to  ECOEs  to  make  them 
more  useful  resources  to  the  SGI.  Over  time,  lessons  learned  and  best  practices  can  be  captured 
in  “train  the  trainer”  packages  for  ECOEs  and  SGIs. 

MOEs  typically  are  difficult  to  obtain  if  the  intent  is  to  identify  the  impact  of  training  or 
education  on  subsequent  behavior  or  perfonnance  (e.g.,  in  a  job,  assignment  or  duty  position). 
There  are  two  ways  that  web-based  utilization  of  ECOEs  could  help  identify  the  efficacy  or 
utility  of  learning.  First,  ECOEs  are  more  likely  to  be  able  to  appreciate  the  implications  of  a 
student’s  knowledge  or  thinking  for  performance  in  theater,  that  is,  to  assess  the  potential  impact 
of  learning.  Second,  former  students  who  have  experienced  the  web-based  dialog  with  ECOEs  in 
a  blended  learning  environment  may,  themselves,  become  ECOEs.  This  would  provide  valuable 
direct  feedback  on  the  actual  impact  of  learning. 

In  summary,  our  work  to  date  on  tools  and  methods  to  close  the  gap  between  the 
classroom  and  the  operational  environment  appear  to  provide  promising  opportunities  to  address 
learner-centered,  knowledge-centered,  and  assessment-centered  needs  of  an  effective  learning 
environment.  These  preliminary  conclusions  are  supported  by  generally  enthusiastic  responses  of 
users  and  stakeholders  in  both  the  Generative  and  Operational  Force. 
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FORMAL  DEVELOPMENT  OF  A  NEW  CAPABILITY 


Critical  Considerations  for  Transition 

Transition  of  an  idea  or  innovation  to  an  environment  of  use  in  the  DoD  depends 
critically  on  systematic  development  of  requirements  that  are  relevant,  effective,  and  reasonable 
with  respect  to  an  anticipated  environment  of  use  (Chairman  of  the  Joint  Chiefs  of  Staff,  2004; 
DAU,  2003).  The  DoD  acquisition  community  depends  on  the  DoD  user  community  for  the 
development  of  such  rational  and  useful  requirements.  The  DoD  science  and  technology  (S&T) 
community  can  play  a  critical  role  in  helping  develop  requirements  that  bridge  the  gap  between 
user’s  statements  of  need,  their  expectations  about  solutions,  and  specifications  with  which 
capability  developers  and  providers  can  work.  The  S&T  community  also  can  play  a  valuable  role 
by  ensuring  that  requirements  are  traceable  back  through  the  process  by  which  they  were 
developed.  This  is  critically  important  to  developers  and  providers  who  may  have  to  re-examine 
assumptions  as  the  application  environment  changes  over  the  life  cycle  of  a  capability.  Toward 
that  end,  we  conceptualized  and  approached  our  R&D  in  terms  of  a  framework  that  is  familiar  to 
the  DoD  acquisitions  community. 

Programmatic  risk  reduction  is  an  important  objective  of  our  pre-systems  acquisition 
strategy  for  developing  a  computer-mediated  learning  environment  to  foster  JEM.  Systematic 
development  of  traceable  requirements  is  one  aspect  of  this  strategy.  Another  aspect  is 
involvement  of  potential  users  and  stakeholders  throughout  the  period  of  performance  for  the 
R&D — to  develop  a  habitual  relationship  with  individuals  in  a  plausible  environment  of  use.  A 
third  aspect  is  to  leverage  existing  capabilities  and  lessons  learned,  most  notably  in  the 
commercial  sector.  This  is  an  established  practice  in  rapid  acquisitions  with  which  several 
members  of  our  team  have  considerable  experience. 

Best  Practices  in  Usability  Engineering 

The  software  market  is  increasingly  a  dynamic  and  unpredictable  environment.  So-called 
“Web  2.0”  products  typically  morph  into  applications  and  patterns  of  use  that  are  not  entirely 
anticipated  and  in  some  cases  quite  surprising.  “Continuous  Beta”  (spiral  development)  has 
become  the  mantra  of  software  development.  This  reality  demands  persistent  contact  with  end 
users  and  stakeholders  to  be  able  to  trace  the  evolution  implicit  in  the  uses  of  a  new  product  or 
service,  to  adapt  what  one  comes  to  understand  are  the  incidental  features  of  a  product,  to  come 
to  a  deeper  understanding  of  the  unique  and  essential  attributes  of  the  product,  and  to  ensure  that 
modifications  and  upgrades  preserve  the  unique  and  essential  attributes  over  the  life  cycle  of  the 
product.  Usability  engineering  has  been  rapidly  co-evolving  with  the  climate  of  spiral 
development  as  lessons  are  being  learned  about  how  to  trace  and  guide  the  evolution  of  a 
software  product.  A  key  consideration  is  how  to  maintain  constant  stakeholder  involvement 
without  being  intrusive,  to  achieve  evidence-based  design  that  captures  the  breadth  of  experience 
with  the  product,  and  that  enables  rigorous  tracking  of  lessons  learned  and  associated  design 
modifications  based  on  such  infonnation  about  use  (Beale,  Courage,  Hammontree,  et  ah,  2008; 
Carrol  &  Rosson,  2007;  Friedman,  Kahn,  &  Borning,  2009;  Kantner,  Sova,  &  Anschuetz,  2005; 
Reddish,  2007). 
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Use  of  multiple  methodologies  including  qualitative  inquiry,  as  in  our  work,  is  considered 
a  best  practice  in  usability  engineering.  Crystallization  and  constant  comparison  are  the  essential 
analytical  tools.  Crystallization  involves  finding  fit  and  coherence  in  the  evidence  from  users 
about  the  utility  and  value  of  an  evolving  design.  It  helps  identify  unintended  consequences  of 
design  particular  features,  identify  subtle  functional  interactions  among  features,  and 
differentiate  the  essential  from  the  incidental.  Constant  comparison  involves  crosschecking 
among  the  interpretations  of  different  sources  of  evidence  and,  most  notably  in  our  work,  vetting 
the  interpretations  with  end  users  from  whom  design-relevant  evidence  was  gathered.  This 
holistic  process  enables  one  to  find  value  in  small  amounts  of  evidence  that  often  are  the  only 
kinds  of  evidence  that  can  be  gathered  in  usability  engineering.  The  whole  is  greater  than  the 
sum  of  the  parts.  Consistent  with  best  practices  in  usability  engineering,  we  utilized  a  format  for 
concisely  capturing  requirements  implicit  in  the  experience  and  opinions  of  end  users  and 
stakeholders.  In  particular,  a  “quad  chart”  was  developed  after  each  significant  interaction  with 
such  informants  (see  e.g.,  Figure  2).  The  quad  charts  subsequently  were  shared  with  the 
informants  to  check  accuracy  and  priority  of  the  interpretations  and  actionable  recommendations. 
Value  added  by  this  practice  is  that  it  shows  infonnants  that  their  time  was  not  wasted;  it  shows 
them  that  the  software  developers  were  listening  to  what  they  had  to  say;  and  it  reveals  the 
potential  impact  of  their  time  and  their  opinions  in  the  requirements  capture  process. 


•v  Socrates'  Window 


bringing  the  COE  into  the  Classroom 


Meetings  at  MC3  Week  of  27  Nov  2006 


Personnel  with  whom  we  interacted 

>  Director  of  CATD 

>  SGI  (observed  class) 

>  SGI  (observed  class) 

>  SGI  (Team  Chief) 

>  SGI  (Module  Director) 

Key  points  from  MC3  personnel 

>  COIN  module  will  be  expanded 

>  SGI  rotations  at  CTCs  every  90  days 

>  Feedback  from  Commanders  through 
Quality  Assurance  Office 

>  MC3  interaction  with  personnel  in  Theater 
can  be  a  win-win  situation 

>  Important  for  students  to  understand 
repercussions  of  command  decisions 

>  COIN  environment  more  relevant  than  COIN 
operations  at  tactical  level 

>  Continuity  across  rotations  (RIP-TOA)  is  a 
major  factor  for  effectiveness — Unity  of 
Effort 


Key  points  from  our  observations 

>  Experts  in  the  COE  (ECOE)  can  fit  into  small 
group  teaching  methods  in  MC3 

>  Interplay  between  general  principles  and 
specific  details  is  critical 

>  ECOE  can  help  students  avoid  assumptions 
about  “school  solutions” 

>  General  principles  help  connect  COIN 
module  with  other  modules  in  MC3 

Next  steps 

>  Explore  opportunities  to  blend  distance 
learning  with  learning  in  MC3 

>  Vet  ideas  with  SGIs  about  how  they  can 
control  ECOE  as  a  resource 

>  Sit  in  on  expanded  COIN  module  in  March 
2007 

>  Review  coordination  between  Fort  Benning 
and  Fort  Knox 

>  Facilitate  meeting  between  CATD  (MCCC) 
and  AWG 


The  Wexford  Group  International  -  Proprietary  Information 


15 


Bringing  the  COE  into  the  Classroom 


Wt  Socrates'  Window 


Interview  with  Chief  of  Tactics  11  November  2009 


Key  points  about  Chief  of  Tactics 

>  Served  as  Chief  of  Tactics,  CATD  JUN  07-JUN  09 

>  Supervised  SGIs  for  the  MCCC 

>  Responsible  for  the  MCCC  Program  of  Instruction 
(POI) 

Key  points  from  Chief  of  Tactics 

>  Assumptions: 

>  MCCC’s  primary  purpose  is  to  train  Company 
Commanders  for  the  BCTs 

>  Second  purpose  is  to  train  Operations  (AS3)  Officers 
for  Battalions  and  BCTs 

>  MCCC  needs  to  help  Captains  learn  how  to  do  in-depth 
analysis,  not  just  the  quick  FRAGO 

>  Nature  of  protracted  warfare  requires  Army 
leadership  to  stop  “micro-management”  of 
Companies  and  Platoons 

>  Incorporation  of  a  “Total  Government  Approach”  is 
needed 

>  Allow  military  to  focus  on  Security  Operations 

>  SW  helps  train  the  Inter-Agency  as  well  as  Military 

>  TDEs  are  essential  to  learning,  especially 
remediation 

>  Real  world  examples  are  available  and  powerful 

>  Ambiguity,  no  one  right  answer  makes  best  TDE 


Key  points  from  our  observations 

>  Outside  Experts  key  to  the  development  of 
analytical  skills  outside  of  basic  Infantry  skills 

>  For  immediate  impact,  it  is  not  enough  to  train 
Captains  in  JEM;  Senior  Leadership  must  be  “on 
board” 

>  SW  can  facilitate  development  across  Joint,  Inter- 
Service,  Inter-Agency,  Multinational  (JIIM) 
Operations  and  Environments 

>  Stories  (TDEs)  build  on  what  you  know;  help  to 
identify  what  you  don’t  know 

>  Open-ended  stories;  many  possible  decisions  and 
outcomes 


Next  steps 

>  Show  the  "two-way”  value  of  SW  to  train  the 
Experts  as  much  as  the  Students 

>  Develop  Stories  (TDEs)  the  facilitate  development 
among  all  parts  of  a  JIIM  operation  or  environment 

>  Goldwater-Nichols  Act  needed  for  the  Inter-Agency 


The  Wexford  Group  International  -  Proprietary  Information 


Figure  2.  Two  of  the  quad  charts  generated  over  the  three-year  period  of  habitual 
interactions  with  end  users  and  stakeholders. 
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SYSTEM  DEVELOPMENT 


Operational  Architecture 

The  most  fundamental  insight  about  the  operational  architecture  (Figure  3),  in  the  context 
of  patentability,  is  that  the  unique  and  essential  attributes  of  Socrates  Window  provide  outside 
experts  with  sufficient  visibility  into  the  course  to  enable  opportunities  for  influence  in  the 
context  of  mentorship  (i.e.,  one  who  deeply  knows  a  learner  and  is  committed  to  development  of 
the  learner). 


Figure  3.  Operational  architecture  for  the  Socrates  Window. 

Capabilities  Integration 

Users  will  use  what  they  can  use,  especially  what  is  at  hand,  and  will  not  let  product 
manufacturers  or  purveyors  dictate  to  them  how  and  when  they  should  do  something.  Users 
cobble  together  an  off-the-shelf  bundle.  Tracking  this  doesn’t  require  formal  system-of-systems 
integration  but  it  reveals  the  potential  family-of -systems  integration  that  can  be  done  on  the  fly 
by  end  users  (Figure  4).  This  fact  of  use  should  be  considered  explicitly  in  the  requirements 
elicitation  process  with  stakeholders  and  end  users. 

The  viability  and  value  of  the  core  concept  has  not  changed  over  the  course  of  the 
project.  Due  diligence  on  the  rapidly  evolving  capabilities  of  Web  2.0  continue  to  show  that  the 
operational  architecture  for  Socrates  Window  addresses  a  gap  in  current  web-based  learning 
and  knowledge  sharing. 
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Figure  4.  Highly  stylized  depiction  of  the  bundling  of  an  array  of  different  products  by 

end  users  based  on  convenience,  personal  preferences,  or  organizational  demands. 

Spiral  Development 

The  development  team  maintained  a  habitual  relationship  with  end  users  over  three  years 
including  two  changes  in  command  as  well  as  changes  in  the  SGI  cadre.  We  developed  a  process 
by  which  we  could  (a)  maintain  this  relationship,  (b)  frequently  elicit  feedback,  and  (c)  could  be 
responsive  without  being  intrusive  to  the  SGI  cadre  or  their  chain  of  command.  The 
considerations  that  drove  the  development  of  this  process  include: 

•  Establish  rapport  and  trust  with  a  group  of  end  users  to  get  visibility  into  their  operating 
environment  and  to  have  opportunities  for  influence  on  it. 

•  Visibility  and  opportunities  for  influence  enable  the  identification  of  implicit  needs  as 
well  as  explicit  requests  for  new  capabilities. 

•  Provide  liaison  between  development  team  and  end  users  to  minimize  demands  on  end 
users  and  to  maximize  impact  of  interactions  with  end  users. 

•  Employ  systems  engineering  process  including  key  personnel  on  development  team  to 
maintain  focus  and  adaptability  as  understanding  of  needs  evolve. 

•  Employ  collaborative  decision-making  by  user  liaison,  programmers,  project 
management,  and  quality  assurance  to  maximize  value  for  end  user. 

•  Two  levels  of  quality  assurance  increase  the  likelihood  that  interactions  with  end  users 
are  positive  and  successful. 

•  Site  visits  are  necessary  but  not  the  only  way  to  maintain  connections  between 
development  team  and  operating  environment  of  end  user. 
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The  process  is  depicted  in  Figure  5.  An  important  element  of  this  process  is  a  tiered 
organization  of  the  development  team  that  insulates  end  users  and  stakeholders  from  unnecessary 
detail,  jargon,  and  esoteric  concepts  of  the  development  process  yet  keeps  scientists  and 
engineers  grounded  in  the  evolving  needs  (and  the  evolving  understanding  of  needs)  of  end  users 
and  stakeholders. 


Figure  5.  Tiered  organization  of  the  development  team  and  the  associated  elements  of  the 
software  development  process. 

Formal  Systems  Engineering 

A  software  systems  engineer  was  retained  to  help  achieve  a  rigorous  systems-engineering 
process  of  appropriate  size  and  level  of  detail  for  the  project.  Part  of  the  guidance  was  to  utilize 
a  formal  systems  engineering  tool  (“Lighthouse”).  Various  views  in  the  Lighthouse  portal  are 
shown  in  Figures  6-10.  The  portal  ensured  tight  coordination  in  the  development  team  as  the 
software  was  developed.  The  software  development  process  was  part  of  the  overall  spiral 
development  in  this  project.  During  the  formal  tracking  of  software  development,  there  was 
continual  and  intense  use  of  the  tool  by  members  of  the  team  on  behalf  of  the  MC3  SGIs.  In 
particular,  documents  that  were  loosely  organized  independently  by  a  number  of  SGIs  were 
uploaded  and  organized  in  Socrates  Window.  This  direct  experience  with  various  versions  of  the 
prototype  enabled  continuous  usability  engineering  to  maximize  the  utility  of  the  tool. 
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Figure  6.  Tracking  versions  in  Lighthouse. 
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Figure  7.  Traceable  activities  of  development  team  in  Lighthouse. 
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The  objectives  of  the  development  process  were: 

•  Hone  in  on  the  most  simple  and  effective  design  of  the  interface  with  respect  to  providing 
outsiders  with  visibility  and  opportunities  for  influence. 

•  Intense  use  of  the  Socrates  Window  to  embed  team  in  the  organization  of  MC3  materials 
and  prepare  interface  for  use  by  the  MC3  SGIs. 

•  Refine  user  needs,  based  on  actual  use  of  the  Socrates  Window  prototype,  reviewed 
through  change  control  before  modifications  implemented  in  accordance  with  associated 
change  proposals. 

•  Stabilize  the  design  through  formal  systems  engineering  configuration  management  that 
stops  churn  on  requirements  and  helps  manage  scope. 
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Figure  8.  Tracking  potential  software  improvements  in  Lighthouse. 
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THE  INNOVATION 


Systems  Engineering  Description  of  Innovation 

A  software  systems  engineer  was  retained  to  develop  an  externally  usable  description  of 
the  innovation  after  two  years  of  concept  refinement  and  system  development  based  on 
documentation  of  the  R&D  to  that  point  in  time  and  based  on  supplementary  interviews  with  the 
development  team. 

JEM 


The  R&D  for  JEM  sets  out  to  provide  a  pedagogical  framework  to  meet  the  needs  of  the 
expeditionary  force.  JEM  has  the  pedagogical  aim  of  teaching  how  the  collection,  exploitation 
and  distribution  of  credible,  valid  and  relevant  intelligence  (knowledge)  can  be  performed.  One 
of  the  tenets  of  JEM’s  pedagogical  framework  is  the  use  of  Socratic  dialogue. 

R&D  seeks  to  determine  the  components  of  the  constructs  associated  with  JEM,  the 
fundamental  attributes  embodied  therein,  the  operational  capabilities  to  which  it  is  relevant,  and 
the  environments  suitable  to  enhance  ground  component  Expeditionary  Mindset  and  the 
associated  Joint  capabilities1.  This  will  be  achieved  through  the  development  of  a  computer- 
mediated  training  environment  that  can  be  used  to  prepare  ground  component  forces  with  the 
necessary  cognitive  skills  for  the  emerging  challenges  of  a  Joint  and  Expeditionary  force. 

The  key  challenges  associated  with  JEM  are: 

•  Identification  of  the  attributes  of  JEM  that  allow  ground  forces  to  counter  the  threats  in 
the  COE; 

•  Closing  the  gap  between  the  COE  and  programs  of  training  and  education; 

•  Providing  users  and  stakeholders  with  visibility  into  and  opportunities  for  influence  on 
development  of  the  innovation; 

•  Assuring  transition  of  the  innovation  into  a  specific  program  of  training  or  education. 

JEM  is  able  to  leverage  a  number  of  unique  opportunities  to  provide  an  environment 
within  which  the  problem  can  be  more  clearly  characterized,  the  key  challenges  met  and  a 
solution  developed,  including: 

•  Currently  there  is  an  unprecedented  abundance  of  expertise  in  expeditionary  operations 
such  as  those  associated  with  COIN; 

•  The  MC3  has  been  updating  its  program  of  instruction  based  on  guidance  from  TRADOC 
to  emphasize  challenges  of  COIN  in  the  COE,  and  to  balance  offensive,  defensive, 
stability  operations; 

•  There  is  rapidly  increasing  interest  and  experience  in  Theater  and  in  CONUS  with  web- 
based  methods  of  sharing  information  and  lessons  learned  about  the  COE. 


1  From  2005  solicitation  of  OSD  under  the  Small  Business  Innovation  Research  program. 
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The  solution  envisaged  for  the  JEM  research  project  was  an  innovative  integrated  set  of 
products  and  processes  which  make  ECOEs  accessible  to  students,  helpful  to  instructors,  and 
pedagogically  prepared  to  participate  in  Socratic  Distance  Learning.  The  set  of  products  and 
processes  comprise  of: 

•  An  approach  which  blends  classroom  learning  with  web-based  learning  and  is 
pedagogically  well  integrated  into  an  established  and  relevant  curriculum; 

•  A  web-based  capability  which  enables  ECOEs  to  help  instructors  implement  guided 
experiential  learning  in  an  established  curriculum; 

•  A  web-based  capability  that  leverages  best  practices  and  lessons  learned  in  knowledge 
databases  and  live  on-line  (i.e.,  synchronous)  discussion  in  Theater  and  in  the  classroom. 

JEM  deals  with  the  issues  associated  in  providing  training  and  education  for  an 
expeditionary  force  that  must  prepare  for  combat  and  non-combat  operations  while  learning  to 
improvise  and  adapt  to  constantly  changing  threats. 

While  the  expeditionary  force  acknowledges  that  no  one  can  ever  be  prepared  enough,  it 
can  still  prepare  for  a  lack  of  preparation.  Education  is  key  tool  in  preparedness.  Traditional 
education  sees  knowledge  flowing  from  the  top  down.  However  in  reality,  knowledge  flows 
from  many  directions.  Students  need  to  learn  how  to  gather  knowledge  from  multiple  sources. 
Instructors  need  to  learn  how  to  identify  the  multiple  sources.  Both  students  and  instructors  need 
to  learn  how  to  disseminate  which  of  the  information  flows  contain  relevant  infonnation.  They 
must  leam  to  leam.  Given  Digby  ’s  Paradox',  both  students  and  instructors  need  to  develop  the 
skills  necessary  to  educate  themselves  and  others  to  meet  the  needs  of  the  expeditionary  force. 

Background 

The  COE  is  the  phrase  given  to  the  environment  the  military  finds  itself  operating  in  at 
any  given  moment.  Historically  the  COE  was  perceived  to  be  a  non-urban  location  (the  plains  of 
Europe,  the  deserts  of  North  Africa,  the  jungles  of  Southeast  Asia)  against  another  formal, 
standing,  regular  military  force  with  both  sides  conducting  traditional  symmetrical  warfare.  The 
last  few  decades  have  seen  the  actual  COE  change  to  include,  more  often  than  not,  an  urban 
location  against  an  irregular  military  force.  Both  sides  are  now  engaged  in  an  asymmetric 
conflict. 

Tactics  and  procedures,  often  developed  in  the  field,  have  allowed  the  regular  military  to 
engage  and  defeat  irregular  forces.  However  the  irregular  forces  change  their  tactics  on  a 
continuous  basis.  Development  of  new  tactics  and  procedures  in  the  field  has  allowed  the 
conventional  military  to  remain  in  step  with  the  evolution  of  the  opponent’s  tactics. 

The  conventional  military  is  a  large  organization  with  a  formal  structure.  The  traditional 
military  training  and  education  establishments  teach  in  accordance  with  the  needs  of  current 
doctrine.  Doctrine  is  detennined  centrally  within  the  military  hierarchy.  While  the  situation  in 


2  Digby’s  Paradox:  While  data  quantity  and  data  delivery  bandwidth  increases,  societal  expectations  are  that  the 
time  available  to  perform  tasks  decreases.  As  a  consequence,  knowledge  must  be  assimilated  in  smaller  units  of 
time  and  therefore  useful  knowledge  becomes  harder  to  obtain. 
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the  COE  is  used  in  the  determination  of  doctrine,  the  view  observed  by  the  hierarchy  is  only  a 
snapshot  in  time  for  all  the  obvious  reasons  of  management.  By  the  time  the  snapshot  of  the 
COE  has  been  used  to  update  doctrine,  the  new  doctrine  used  to  developing  teaching  and 
teaching  delivered,  an  amount  of  time  will  have  passed.  As  a  consequence,  the  students 
emerging  from  the  educational  process  will  enter  a  COE  that  bears  no  resemblance  to  the  one 
they  were  taught  about.  As  a  consequence,  additional  knowledge  must  be  passed  on  to  them 
once  they  are  in  the  COE. 

The  time  delay  in  recycling  knowledge  from  the  COE  back  into  the  education 
environment  is  most  pronounced  in  COIN  operations.  Also  recent  years  have  seen  increased 
military  involvement  in  Operations  Other  Than  War  (OOTW).  This  is  still  a  changing 
environment  with  an  evolving  list  of  players  including  other  governmental  agencies  (OGA)  and 
non-governmental  organizations  (NGO). 

The  knowledge  gap  due  to  the  time  delay  for  COIN  will  revolve  around  the  current 
tactics  and  procedures  used  by  both  sides  in  the  COE.  For  OOTW  the  knowledge  gap  will 
revolve  around  the  capabilities  that  OGA  and  NGO  bring  to  the  COE.  In  both  cases  local 
cultural  knowledge  about  the  COE  may  be  missing  from  the  formal  training. 

The  Educational  Setting 

Formal  continuing  professional  development  is  very  much  part  of  military  doctrine.  A 
military  unit’s  operational  state  is  juxtaposed  with  periods  of  exercise  and  training.  Individuals 
in  the  military  are  trained  and  educated  as  they  are  promoted.  Much  of  an  individual’s  education 
takes  place  in  formal  educational  establishments.  Often  the  emphasis  is  on  small  group 
instruction  with  a  significant  amount  of  interpersonal  interaction.  A  consequence  of  this  is  a 
geographically  constrained  community,  which  while  open  to  insiders,  is  difficult  for  outsiders  to 
gain  access  to. 

Initial  Vision 

The  present  research  investigated  an  operational  architecture  to  allow  the  reduction  in  the 
time  delay  for  knowledge  transfer  from  the  COE  to  the  classroom  (see  Figure  3).  The  proposed 
operational  architecture  allows  knowledge  (hereafter  collectively  referred  to  as  knowledge)  to  be 
transferred  from  the  COE  to  the  educational  environment  without  it  following  the  chain  of 
command.  A  peer-to-peer  structure  is  used  to  implement  the  operational  architecture.  The 
operational  architecture  relies  on  ECOEs  to  communicate  knowledge  to  the  instructors  in  the 
educational  environment.  The  experts  would  be  from  either  a  military  organization,  OGO  or 
NGO. 


Development  of  the  operational  architecture  revealed  two  needs:  in  order  to  be  effective 
the  expert  would  need  visibility  of  the  course  content  in  order  to  ground  their  knowledge  in  the 
appropriate  pedagogical  context;  those  offering  expertise  need  to  be  assessed  to  ensure  they  are 
bono  fide  and  able  to  provide  relevant  knowledge. 
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In  essence  the  operational  architecture  is  for  the  provision  of  visibility  of  an  insular 
domain  to  outside  experts,  in  order  to  increase  the  tempo  of  knowledge  exchange.  The 
operational  architecture  allows  for  social  interaction  in  the  education  domain. 

Just  as  the  educational  establishment  has  its  own  ways  of  working,  the  expert’s  own 
organizations  will  have  their  ways  of  doing  business.  Therefore  a  solution  cannot  be  prescribed 
across  multiple  organizations.  The  solution  must  be  flexible. 

Major  Use  Cases 

Provide  Visibility  of  Course  Content.  It  is  comparatively  easy  for  a  person  to 
communicate  knowledge  to  others.  However  for  that  knowledge  to  be  effective  and  provide 
added  value  to  the  recipient,  the  knowledge  should  be  delivered  with  reference  to  the  context 
within  which  it  will  be  used  by  the  recipient.  Therefore  it  was  recognized  that  the  provision  of 
expertise  would  be  more  effective  if  the  expert  was  able  to  understand  the  context  within  which 
their  knowledge  would  be  used  and  what  had  led  to  the  need  for  that  knowledge.  The  course 
objectives  and  the  course  content  (slides,  reading  lists,  documents  and  examples)  go  a  long  way 
to  establishing  the  context  within  which  the  expertise  will  be  used.  Visibility  of  course  content 
was  also  important  to  allow  experts  who  had  no  knowledge  of  the  course,  or  its  structure,  or 
educational  techniques,  to  contribute. 

Provide  Expert  Selection.  While  every  person  working  within  an  environment  will  gain 
knowledge  about  that  environment,  it  does  not  necessarily  imply  that  that  knowledge  will  be 
relevant  to  others  or  that  it  can  be  delivered  in  a  meaningful  manner.  As  a  consequence,  it  is 
necessary  to  assess  those  offering  their  services  as  an  expert.  Rather  than  use  the  traditional 
recruiting  cycle  where  the  expert  would  prepare  a  resume  that  would  be  reviewed  by  the 
instructors,  another  approach  was  considered  that  would  allow  the  instructors  to  ask  the  experts 
focused  questions  and  then  review  the  responses. 

Refining  the  Operational  Architecture 

Improving  the  visibility  to  ECOEs  also  meant  that  visibility  of  the  material  to  insiders 
such  as  the  instructors  and  students  could  also  be  improved.  However  simply  providing 
visibility  to  the  course  content  in  its  existing  silos  would  not  be  an  effective  mechanism  to  aid 
understanding  for  anyone.  A  method  was  needed  for  categorizing  the  content  that  implicitly 
showed  the  interconnections  between  categories.  As  an  educational  establishment  operates  on  a 
calendar,  it  was  also  important  to  tie  the  content  to  its  place  in  the  academic  year.  This  would  be 
an  aid  for  both  students  and  instructors  wanting  to  rapidly  locate  content  associated  with  a 
particular  teaching  period.  It  also  would  provide  outside  experts  with  visibility  into  the  temporal 
context  for  queries  and  needs,  that  is,  issues  of  the  day  as  well  as  considerations  in  immediately 
prior  and  subsequent  lessons. 

It  was  recognized  that  improving  visibility  also  involved  breaking  down  the  “walls” 
around  the  education  establishment  to  the  experts  who  were  geographically  dispersed  to  gain 
visibility.  A  side  effect  of  this  would  be  to  allow  students  to  have  visibility  of  material  while  not 
at  the  education  establishment. 
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As  well  as  being  geographically  dispersed,  experts  are  likely  to  be  located  in  different 
time  zones.  Therefore  mechanisms  for  communication  must  be  able  to  cope  with  this. 

Systems  Engineering  Lifecycle 

A  spiral-based  systems  engineering  life  cycle  has  been  adopted  for  the  JEM  program.  This 
incremental  and  iterative  development  method  offers  a  number  of  benefits  for  the  JEM  program: 

•  The  system-of-interest  is  open  to  the  insertion  of  new  technologies.  This  is  especially 
important  with  respect  to  web-based  technologies  where  their  development-cycle  time  either 
equals  or  is  less  than  the  length  of  the  JEM  program. 

•  A  candidate  system-of-interest  can  be  rapidly  developed  and  then  assessed  against  the 
emerging  user  requirements. 

•  The  user  requirements  can  be  gathered  and  validated  as  and  when  opportunities  arise.  The 
validated  requirements  can  then  be  fed  into  the  development  process  when  they  are  available. 
This  is  more  beneficial  than  having  a  large  requirement  to  capture  activity  at  the  beginning  of 
the  program  which  results  in  a  static  requirement  set  for  the  duration  of  the  program. 

•  The  development  spiral  allows  for  the  development  candidate  systems-of-interest  in  phase 
with  the  funding  cycle. 

•  The  iterative  introduction  of  candidate  systems-of-interest  into  the  operational  enviromnent 
allows  requirements  of  increased  focus  and  fidelity  to  be  captured  on  each  iteration. 

Integration,  Verification  and  Validation  Strategy 

The  integration  strategy  for  the  JEM  research  project  allows  for  the  integration  of 
technology  solutions  with  increasing  fidelity  in  each  iteration.  For  example,  the  Phase  1 
candidate  system-of-interest  leveraged  Web  1.0  technologies  in  order  to  draw  out  issues  and 
constraints  associated  with  the  use  of  Web  technologies  within  the  educational  environment. 

The  next  iteration  of  the  candidate  system-of-interest  will  use  “Web  PA”  technologies  (a  subset 
of  Web  2.0  technologies)  that  will  be  chosen  to  implement  the  requirements  validated  against  the 
Web  1.0  candidate. 

The  validation  strategy  uses  the  iterative  placement  of  candidate  systems-of-interest  into 
the  education  environment  to  provide  a  means  by  which  the  suitability  of  the  original 
requirements  can  be  measured  (and  hence  validated).  The  validation  of  the  requirements  with  a 
candidate  system-of-interest  in  its  operational  environment  allows,  not  only  the  requirements  for 
the  system  to  be  validated  but  also  requirements  for  the  educational  activities  can  be  captured  as 
the  impact  of  the  candidate  system-of-interest  becomes  more  obvious.  This  allows  for  smooth 
integration  of  the  candidate  system-of-interest  into  the  operational  environment. 


Socrates  Window 

A  number  of  different  approaches  are  available  to  implement  the  operational  architecture. 
The  team  chose  to  develop  a  prototype,  Socrates  Window,  instead  of  purchasing  commercial-off- 
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the-shelf  (COTS)  software  that  would  be  more  difficult  to  tailor  to  user  needs.  Socrates  Window 
provides  visibility  to  the  course  content  through  web  access  to  a  document  repository.  The 
documents  within  the  repository  are  categorized  using:  an  ontology  based  on  the  hierarchy  of 
military  doctrine;  a  course  syllabus;  and  course  calendar.  Socrates  Window  also  provides  an 
email-style  messaging  service  to  allow  “time-displaced”  communication. 

Socrates  Window  is  a  product  to  support  R&D  in  the  JEM  project.  One  of  the  tenants  of 
JEM’s  pedagogical  framework  is  the  use  of  Socratic  dialogue.  A  goal  of  Socrates  Window  is  to 
provide  a  means  of  distance  learning  through  a  web-based  Socratic  dialog.  Socrates  Window  is 
to  support  JEM’s  pedagogical  aim  of  teaching  how  the  collection,  exploitation  and  distribution  of 
credible,  valid  and  relevant  knowledge  can  be  performed.  As  the  research  project  is  investigating 
a  number  of  methodologies  that  will  help  solve  the  pedagogical  needs  of  the  expeditionary  force, 
the  implementation  of  Socrates  Window  needs  to  be  flexible  to  allow  the  insertion  (or  removal) 
of  features  that  support  one  or  more  of  JEM’s  teaching  strategies. 

Identification  of  Emergent  Properties 

The  introduction  of  candidate  systems-of-interest  into  the  educational  environment  and 
the  subsequent  monitoring  of  their  use  allows  for  the  identification  of  emergent  properties.  The 
emergent  properties  may  manifest  themselves  as  unforeseen  but  beneficial  uses  of  the  system  or 
changes  in  the  ways-of-working  for  the  instructors,  experts,  or  students.  The  iterative 
development  cycle  allows  the  emergent  properties  to  be  identified  and  subsequently,  if 
necessary,  modifications  to  the  system’s  requirements  can  be  made  in  order  to  enhance  the 
emergent  properties. 

Information  Architecture 

Ultimately  the  JEM  research  project  is  about  the  provision  of  information,  in  this  instance 
to  students  from  their  instructors  and  associated  experts.  Interactions  with  users  and  stakeholders 
in  the  context  of  the  candidate  system-of-interest  provided  evidence  that  the  appropriate 
management  of  information  is  key  to  the  success  of  the  JEM  program.  Further  studies  on  the 
information  architecture  are  being  conducted.  These  include  identification  of  appropriate  meta¬ 
data  for  classifying  and  categorizing  data,  including  the  use  of  folksonomies  and  ontologies. 

Security  Architecture 

In  addition  to  the  “traditional”  information  technology  security  measures  for  a  military 
system,  the  R&D  identified  the  need  for  those  interacting  with  the  system  to  be  assigned  “roles.” 
At  the  broadest  level  the  roles  are  instructor,  student  and  expert.  However  initial  findings 
indicate  that  the  implementation  of  roles  needs  to  be  in  a  loosely  restrictive  manner.  Further 
research  is  being  conducted  in  this  area. 

In  addition  to  the  assignment  of  roles,  the  R&D  recognized  the  need  for  assurance  as  to 
the  identity  of  the  people  interacting  with  the  system,  in  particular  the  experts.  JEM  and  its 
system-of-interest,  Socrates  Window,  change  the  educational  environment  from  being  a  single 
point  -  the  classroom  -  to  a  distributed  environment.  The  distributed  nature  of  this  environment, 
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in  particular,  where  the  expertise  is  likely  to  be  found,  coupled  with  the  notion  that  an  expert 
may  provide  expertise  to  many  different  “communities”  strains  the  traditional  methods  for 
verifying  and  validating  the  expertise  provide  by  an  expert.  As  a  consequence  further  research  is 
being  conducted  into  digital  identities  and  how  they  may  help  solve  this  problem. 

Communications  Architecture 

At  its  most  fundamental  level,  the  Web  is  simply  another  means  of  communication.  As 
such,  not  only  does  it  provide  communication  but  it  also  relies  upon  communication 
technologies.  As  the  JEM  research  project  is  not  just  the  provision  of  a  software  solution  but 
also  the  provision  of  an  approach  to  classroom  learning,  the  other  alternative  communication 
mechanisms  available  to  support  the  approach  were  also  identified.  Consequently  the 
technologies  required  for  the  Web-based  communication  and  the  alternative  communications 
mechanisms  need  to  be  brought  together  into  a  coherent  communications  architecture.  Further 
research  on  the  communications  architecture  is  to  be  undertaken. 

COTS  Technologies  Identification  and  Assessment 

An  ongoing  project  performed  during  each  iteration  of  the  JEM  research  project  was  the 
identification  of  commercial  off-the-shelf  technologies  that  could  be  leveraged  in  the  provision 
of  the  system-of-interest.  The  research  was  performed  in  two  ways:  First,  COTS  technologies 
and  solutions  which  could  be  used  to  implement  a  particular  function  or  group  of  functions  were 
identified  in  the  system  architecture.  Second,  existing  websites  providing  community-based 
services  were  identified  which  exhibited  one  of  more  relevant  functions.  Then,  if  possible,  the 
components  used  to  implement  these  websites  are  identified. 

Critical  Dependencies  between  Major  Item  Developments 

At  the  end  of  the  project,  dependencies  between  major  item  developments  were 
identified.  Dependencies  between  major  items  in  the  candidate  system-of-interest  and  the 
education  environment  have  also  been  identified. 

Critical  dependencies  have  been  identified  between  the  representation  of  the  syllabus  and 
the  representation  of  a  calendar  for  the  course.  The  subsequent  association  of  documents  with 
the  syllabus  areas  and  calendar  items  has  a  dependency  on  the  representation  of  the  syllabus  and 
calendar.  These  dependencies  likely  impact  the  implementation  of  the  calendar  application 
service  and  document  management  application  service  in  the  system-of-interest.  The  studies  into 
the  information  architecture  and  the  COTS  technologies  identification  and  assessment  will  help 
to  mitigate  risks  associated  with  these  dependencies. 

Conclusions 

Simplicity  is  the  key  but  only  if  it  is  sufficient  to  situate  the  outside  expert  in  the  context 
of  participants  in  the  course  and  enable  outsiders  to  act  somewhat  in  the  role  of  a  mentor. 
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Systems  Engineering:  Operational  Views 


Figure  11.  OV-1  high-level  operational  concept  graphic. 


Figure  12.  OV-2  operational  node  connectivity  description. 


OV-2 
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-Knowledge- 


The  needlines  for  a  pre-JEM  scenario  are  the  same  as  the  post- JEM  scenario.  Pre-JEM 
lines  of  communication  were  via  a  consolidation  and  aggregation  via  the  chain  of  command  in 
order  to  satisfy  the  needlines.  The  post- JEM  scenario  lines  of  communication  map  on  to  the 
needlines.  The  aggregation  of  knowledge  is  reduced  as  communication  is  direct  (or  as  near  as 
possible,  direct). 

Pre-JEM  aggregation  and  consolidation  introduced  delays  into  the  time  taken  to  transfer 
knowledge  -  therefore  the  tempo  is  reduced.  Will  the  post- JEM  reduction  in  aggregation  and 
consolidation  activities  give  rise  to  saturation  of  knowledge  going  to  the  instructor,  especially 
now  the  time  delay  is  reduced  and  the  tempo  increased?  Is  tempo  truly  the  inverse  of  time  delay? 
Time  delay  implies  a  sequence  of  events  whereas  tempo  implies  and  circular  sequence  of  events. 
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Knowledge  on  COE 
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Knowledge,  course  content,  course  context 

Instructor 

Student 

Expert 

Expert 

Question  on  COE 

Question  on  COE 

Instructor 

Expert 

Expert 

Instructor 

Question  on  experience 

Response  on  experience 

Instructor 

Instructor 

Expert 

Expert 

Course  context 

Course  content 

TRADOC 
HQ  COE 
Expert 

Instructor 
TRADOC 
HQ  COE 

Course  objectives,  inc.  doctrine 

Knowledge  on  COE 

Knowledge  on  COE 

Figure  13.  OV-3  operational  information  exchange  matrix. 


Figure  14.  OV-4  organizational  relationships. 
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Figure  15.  OV-5  operational  activity  model. 


OV-5 


The  operational  activities  shown  are  the  primary  activities.  Knowledge  and  experience  is 
gained  as  a  consequence  of  operating  in  the  COE  -  rather  than  having  the  activity  “gain 
knowledge.”  Similarly,  the  instructor  uses  the  knowledge  when  writing  the  course  -  rather  than 
through  the  activity  “use  knowledge.” 

OV-6 


The  OV-6  has  intentionally  been  left  blank  at  this  stage.  Adding  detail  to  the  OV-6  will 
introduce  a  “sequence  of  events”  which  at  this  stage  we  do  not  want  to  stipulate  as  it  may  imply 
a  constraint  on  the  interactions  between  actors. 


Figure  16.  OV-7  logical  data  model. 
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Systems  Engineering:  Use  Cases 


Operator  in  COE 


Figure  17.  Pre-JEM  use  case. 
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Figure  18.  JEM  use  case. 
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Tangible  Prototype:  Socrates  Window 


A  prototype  was  produced  to  guide  the  development  of  requirements  for  the  innovation 
and  to  ground  the  usability  engineering  assessments  of  the  innovation  in  the  context  of  Army 
education.  Simplifying  the  portal  was  a  high  priority  in  order  to  achieve  the  objective  of 
expeditiously  situating  an  outsider  in  the  context  of  a  course  such  as  the  MC3.  Views  accessible 
within  the  portal  are  shown  in  Figures  19-21. 
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Familiar  conceptual  maps  provide  outside  experts  with  easy  and 
rapid  visibility  into  the  context  for  instructor’s  or  student’s 
explicit  requests  for  information  or  implicit  needs  for  expertise 
thus  facilitating  opportunities  for  influence  of  outside  experts 
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Figure  19.  Utilize  a  familiar  conceptual  construct  such  as  the  doctrinal  hierarchy 
of  operations  for  Full  Spectrum  Operations  as  one  of  a  small  number  of  views  as 
points  of  entry  into  the  MC3. 
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Figure  20.  Showing  the  course  calendar  is  a  best  practice  in  web-based  learning. 
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Figure  21.  List  of  documents  used  in  the  course. 
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Best  Practices  in  Selection  of  Elite  Personnel 


A  COTS  solution  is  recommended  for  selection  of  outside  experts.  In  particular,  the  web- 
based  capability  offered  by  AccuHire  Inc.  was  utilized  in  the  JEM  project.  In  addition  to  its  use 
in  the  commercial  sector,  this  portal  and  the  associated  process  have  been  utilized  for  a  number 
of  years  in  selection  of  the  world’s  most  elite  military  personnel.  The  process  utilizes  highly 
targeted  questions  that  get  directly  at  a  candidate’s  actual  experience.  The  questions  and  the  on¬ 
line  answers  the  candidates  provide  are  utilized  in  a  subsequent  behavioral  interview  that  focuses 
simultaneously  on  the  details  and  meaning  of  a  candidate’s  actual  experience. 

»•  'll:1  -'ll1  !*< "  W. !' !  f  .T»wVTf  •»•■•.•!  W  '  '.'T. '  11  ~  1 1  . 4 

Q  •  •  *  •  hops  //aca>«e6  aco>»e  com/aca/reAecnjter/ankantsarTWo  cfm  *  ®  *»  x  muw:  soaJ  sites  | /*_• 


*  «*  H\’  0Logn  0»lP.eoutwTo  *  ft  •  _B  #  *  -rPajs  *  JTocfc* 


Figure  22.  AccuHire  website  prototyped  for  Socrates  Window. 

The  following  needs  for  outside  expertise  in  the  MC3  were  identified: 

•  Interagency  Operations  Advisor 

•  Civil  Affairs  Advisor 

•  Coalition  Military  Advisor 

•  COIN  Operations  Advisor 

•  Legal  (e.g.,  JAG)  advisor 

•  Policing  Operations  (e.g.,  MP)  Advisor 

•  Information  Operations  Advisor 

•  Human  Terrain  Teams  (Religious/Ethnic/Cultural  Advisor) 

•  Human  Intelligence  (HUMINT)  Advisor 

•  Signals  Intelligence  (SIGINT)  Advisor 

•  Engineering  Infrastructure  (e.g.,  municipal  systems)  Advisor 

•  Improvised  Explosive  Device  (IED)  Advisor 

•  Distance  Learning  Expert 
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•  Education  Consultant 


Utilizing  the  AccuHire  process,  position  descriptions  were  developed  for  two  important  areas 
of  outside  expertise: 

Interagency  Advisor 

•  Experience  working  in  the  Inter-Agency  environment,  planning,  executing,  coordinating 
and  synchronizing  the  efforts  of  multiple  government  agencies. 

•  Experience  serving  on  a  Joint  Inter-Agency  Task  Force 

•  Experience  serving  as  a  Liaison  Officer  in  a  government  agency  other  than  your  own 

•  Experience  as  an  exchange  officer  in  a  school  or  assignment  of  a  government  agency 
other  than  your  own 

Civil  Affairs  Advisor 

•  A  Civil  Affairs  Advisor  has  expertise  in  exercising  the  relationship  between  military 
forces  and  the  civil  component,  including  nongovernmental  or  intergovernmental 
organizations  in  areas  where  military  forces  are  present. 

•  The  civil  affairs  advisor  may  also  involve  the  application  of  Civil  Affairs  expertise  in 
areas  normally  the  responsibility  of  the  civilian  government. 

•  Additionally,  the  civil  affairs  advisor  has  experience  with  not  only  tactical  issues,  but  also 
Combat  Support  and  Combat  Service  Support  issues,  host  nation  support  and  care  of 
displaced  civilians.  S/he  has  done  analysis  regarding  the  impact  of  operations  on  public 
order  and  safety,  the  potential  for  disaster  relief  requirements,  noncombatant  evacuation 
operations,  emergency  services,  and  protection  of  culturally  significant  sites. 

•  The  Civil  Affairs  Advisor  provides  feedback  in  how  the  culture  in  the  AO  affects 
operations.  S/he  is  familiar  with  the  civil  considerations  (areas,  structures,  capabilities, 
organizations,  people,  and  events). 

Behavioral  Interview 

Interactions  with  outside  experts  after  use  of  Socrates  Window  led  to  the  conclusion  that 
a  revealing  format  for  the  behavior  interview  could  include  collaborative  development  of  tactical 
decision  exercises  (TDE)  with  outside  experts,  especially  to  the  extent  that  the  TDE  bring  all 
parties  in  the  “interview”  to  the  brink  of  what  is  known  to  them. 
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FOCUS  GROUPS  AND  COLLABORATIVE  REFLECTION  WITH  OUTSIDE  EXPERTS 


This  stage  of  our  usability  engineering  process  was  initiated  once  design  of  Socrates 
Window  was  stabilized  through  systems  engineering  change  control  and  configuration 
management.  Two  major  groups  of  experts  were  utilized  in  usability  trials  with  Socrates 
Window:  (a)  personnel  from  the  Army  Center  for  Enhanced  Performance  (ACEP)  who  had 
relatively  little  knowledge  of  Army  operations,  and  (b)  personnel  associated  with  the  Army  Air 
Defense  community  who  were  knowledgeable  about  Anny  operations  but  from  a  different 
specialty  relative  to  the  MC3.  The  fundamental  query  of  these  outside  experts  was  whether  and 
to  what  extent  Socrates  Window  provides  outsiders  with  expeditious  visibility  into  the  MC3 
sufficient  to  reveal  their  opportunities  for  influence  on  MC3? 

Outside  Experts  from  Army  Center  for  Enhanced  Performance 

There  were  two  objectives  of  this  first  focus  group  with  outside  experts.  First,  we  wanted 
to  assess  the  efficacy  of  Socrates  Window  in  an  extreme  case  of  outside  experts  with  relatively 
little  knowledge  of  Army  operations,  Army  education,  and  the  MC3  in  particular.  Second,  we 
wanted  to  explore  the  technique  of  collaborative  reflection  in  a  focus  group  format  for  a  usability 
engineering  assessment. 

Questions  Addressed 

•  What  degree  of  understanding  of  the  MC3  and  Army  operations  did  ACEP  users  gain 
from  utilizing  Socrates  Window? 

•  What  of  value  to  the  MC3  do  they  think  they  could  contribute  from  their  area  of 
expertise? 

Method 

•  Trials  conducted  in  February  2009 

•  Participants  included  five  Performance  Enhancement  Specialists 

o  Ph.D.  or  M.S.  in  Sports  Perfonnance/Psychology 
o  Experience  working  with  Soldiers  ranged  from  6  months  to  2  years. 

•  Experts  given  some  experience  in  using  Socrates  window 

•  Collaborative  working  group  to  address  utility  of  Socrates  window 

Findings  from  outside  experts 

•  The  outside  experts  quickly  were  able  to  get  oriented  to  the  course  and  identify  places 
where  their  expertise  might  be  useful. 

•  Acronyms  can  be  a  large  inhibitor  to  creating  the  desired  visibility. 

•  An  overview  of  the  arrangement  of  the  course  would  be  helpful  in  orienting  non-military 
experts.  While  the  outcome  goals  listed  for  each  module  are  useful,  a  higher  level 
orientation  that  included  a  description  of  the  students  and  their  experiences,  and  the  intent 
of  the  course,  would  help  outside  experts  operate  more  autonomously  in  detennining 
where  their  expertise  could  be  of  value. 

•  The  technique  of  collaborative  reflection  should  be  developed  further  and  employed  as  a 
means  to  conduct  a  condensed  contextual  inquiry  in  usability  engineering  assessment. 
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Design  implications 


•  SGIs  may  have  to  make  some  additional  effort  to  allow  outside  experts  visibility  into 
their  course.  A  short  description  of  each  class,  and  limiting  the  use  of  acronyms  in  the 
software  tool  would  seem  to  offer  the  greatest  benefit  to  outsiders. 

•  While  these  steps  may  not  be  fully  embraced  by  SGI,  they  will  hopefully  see  the 
additional  benefit  of  better  situational  awareness  by  outside  experts.  Additionally,  they 
will  have  fewer  inquiries  about  what  is  being  taught,  and  more  productive  dialogue  about 
how  to  improve  the  subject  matter. 

Outside  Experts  from  Air  Defense  Community 


Purpose 

Given  rapid  changes  in  the  COE,  along  with  incremental  development  toward  operating 
as  a  more  expeditionary  force,  the  need  is  increasing  for  the  U.S.  Army  warfighter  to  quickly 
recognize,  catalogue,  share  experience,  and  update  classroom  material  to  increase  in  its 
relevance.  This  increase  in  the  rate  of  change  affects  training  requirements  for  both  novice  and 
experienced  Soldiers,  forcing  training  to  incorporate  increases  in  level  of  interaction,  change,  and 
complexity.  Specifically,  these  increases  force  instructors  to  develop  timely  teaching  points  that 
rest  on  solid  doctrinal  foundations  while  addressing  current  issues. 

As  the  primary  goal  of  the  Socrates  Window  conceptual  learning  environment  is  to 
provide  capabilities  that  support  interactions  between  students  and  the  ECOEs,  under  the  control 
of  a  SGI,  the  purpose  of  this  analysis  was  to  further  uncover  measures  of  utility  for  this 
conceptual  program.  Specifically,  this  analysis  was  designed  to  collect  and  codify  some  of  the 
opinions  of  military  SMEs  on  the  fitness  of  the  Socrates  Window  concept  for  use  in  the 
Captain’s  Career  Course  (CCC),  as  well  as  their  determinations  of  the  “practical  value”  of  this 
conceptual  technology  for  use  in  the  CCC  instructional  fonnat. 

Framework  for  Assessing  Value 

Broad  Questions  for  Interview 

(1)  Where  and  how  can  the  MC3  benefit  from  what  you  know  about  your  specialty? 

(2)  Where  and  how  can  the  MC3  benefit  from  what  is  taught  in  your  course? 

(3)  Where  and  how  can  your  course  benefit  from  what  is  taught  in  the  MC3? 

Broad  Questions  for  Interview  with  “Thought  Probes” 

(1)  Where  and  how  can  the  MC3  benefit  from  what  you  know  about  your  specialty? 

•  Thought  Probe(s): 

o  Ask  the  interviewee  to  consider  how  an  Air  Defense  Expert’s  knowledge  of  types 
of  Air  Defense  systems  might  be  useful  to  the  MC3. 
o  Ask  the  interviewee  to  consider  how  an  Air  Defense  Expert’s  knowledge  of  the 
general  system  capabilities  Air  Defense  systems  might  be  useful  to  the  MC3. 
o  Ask  the  interviewee  to  consider  how  gaining  an  understanding  of  Air  Defense 
systems  capabilities  might  be  useful  to  MC3. 
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■  System  capabilities:  traditional  uses 

■  System  capabilities:  non-  traditional  uses 

■  System  limitations 

o  Ask  the  interviewee  to  consider  how  knowing  the  general  requirements  to  employ 
an  Air  Defense  system  (most  efficiently)  might  be  useful  to  MC3. 

■  In  terms  of  “Scheme  of  maneuver”  (Course  of  action) 

(2)  Where  and  how  can  the  MC3  benefit  from  what  is  taught  in  your  course? 

•  Thought  Probe(s): 

o  Ask  the  interviewee  to  identify  the  “top  3  to  5”  most  critical  pieces  of  information 
taught  in  the  Air  Defense  course. 

o  Ask  the  interviewee  to  explain/think  about  how  this  information  can 
improve/enhance  the  MC3. 

(3)  Where  and  how  can  your  course  benefit  from  what  is  taught  in  the  MC3? 

•  Thought  Probe(s): 

o  Ask  the  interviewee  to  consider  their  personal  experience  and  then  name  the  “top 
3  to  5”  most  critical  omissions  from,  or  discrepancies  taught  in  the  Air  Defense 
course. 

o  Ask  the  interviewee  to  explain/think  about  how  adding  this  information  can 
improve/enhance  the  MC3. 

o  (Based  on  their  review  of  the  website,  ask  the  interviewee  to  think  about  the 
information  that  is  taught  in  the  MC3  and  identify  concepts  that  could 
improve/enhance  the  Air  Defense  course?) 


Method 

Interview  participants.  The  SMEs  interviewed  for  this  analysis  were  Air  Defense 
Artillery  (ADA)  Officers  with  a  wealth  of  experience,  ranging  from  12  to  18  years  of  active-duty 
military  service  within  the  U.S.  Anny.  These  knowledge  elicitation  interviews  were  conducted 
on  October  16,  2009.  There  were  two  interview  sessions,  with  two  SMEs  present  in  each  session 
(there  was  a  total  of  4  SMEs  interviewed  for  this  effort).  The  interview  team  consisted  of  four 
interviewers;  with  three  of  these  interviewers  calling  in  from  locations  across  the  country,  and 
the  remaining  interviewer  was  co-located  with  the  SME  in  Colorado  Springs,  CO. 

Procedure  and  analysis.  For  the  current  knowledge  elicitation,  all  observations  were 
conducted  by  a  four-person  interview  team  to  ensure  coverage  and  responsibility  for  each  of  the 
primary  interview  tasks.  These  tasks  include  asking  the  primary  thought  questions,  considering 
question  redirects  and  follow-ups,  taking  notes,  and  keeping  the  discussion  relevant  to  the 
intended  topic.  While  all  members  of  the  interview  team  took  some  notes,  the  primary  note 
taking  responsibility  was  borne  by  one  interview  team  member.  Following  the  interview,  this 
team  member  performed  the  following  actions:  (1)  aggregated  all  of  the  notes  taken  from  each 
interview  session,  (2)  identified  overlapping  content,  (3)  resolved  all  conflicting  infonnation,  (4) 
created  a  primary  interview  ‘notes’  repository,  and  (5)  performed  a  thematic  analysis  of  the 
newly  created  interview  ‘notes’  repository. 
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The  type  of  thematic  analysis  used  in  the  present  research  was  categorical  in  nature. 
Using  the  original  proposal  for  this  technological  concept  evaluation  as  a  guide,  a  series  of 
categorical  themes  were  identified  as  being  critical  assessment  points  in  determining  to  the 
effectiveness  and  applicability  of  the  Socrates  Window  technology.  These  over-arching 
categorizations  were  (1)  the  primary  utility  for  technology  like  Socrates  Window,  (2) 
information  that  can  be  taught/enhanced  with  Socrates  Window,  and  (3)  main  concerns  with  the 
implementation  of  conceptual  learning  environment  technology  like  Socrates  Window.  Once 
these  themes  were  identified,  the  interview  transcripts  were  reviewed  and  the  textual  elements 
that  fit  each  of  these  categorizations  were  extracted  and  aggregated  across  sub-categories. 

Thematic  analysis  of  notes.  The  SMEs  interviewed  for  this  analysis  generally  found  a 
good  measure  of  utility  to  be  associated  with  the  concept  of  the  Socrates  Window  learning 
environment,  in  addition  to  the  technological  implementation  with  which  they  were  presented. 
The  SMEs  explicated  the  applicability  of  the  concept  and  “value  added”  of  this  technology  by 
providing  their  general  impressions  of  the  tool,  considering  the  projected  uses  for  the  tool,  and 
then  considering  situations  (and/or  types  of  information)  for  which  the  tool  could  optimally  be 
used  to  convey  expertise. 

Specifically,  beyond  the  conveyance  of  a  specialty  area  knowledge  (associated  with  a 
mission  occupational  specialty  -  MOS),  the  SMEs  interviewed  for  this  effort  indicated  the 
following  primary  uses  for  the  Socrates  Window: 

•  Unit  Capability  and  Compatibility 

•  Enhance/Encourage  Mission  Adaptability 

•  Encourage  “Out-of-the-Box”  thinking 

•  Encourage  Critical  Thinking 

•  Operational  Considerations/Needs 

•  Tying  doctrine  to  operational  examples 

•  Sharing  OPFOR/insurgent  responses  to  Army  TTPs  taught  in  class 

A  second,  more  detailed  thematic  analysis  was  performed  on  the  transcribed  notes  for  the 
purpose  of  identifying  the  specific  categories  of  infonnation  that  the  SMEs  felt  could  be  taught, 
and  or  enhanced,  through  the  use  of  the  Socrates  Window  learning  environment.  More 
specifically,  this  second  thematic  analysis  was  designed  to  highlight  the  particular  topics,  subject 
areas,  and/or  types  of  infonnation  that  could  be  trained  through  the  use  of  Socrates  Window. 
These  categories  were  identified  from  the  explicit  examples  provided  by  the  interviewed  SMEs, 
as  well  as  through  careful  analysis  of  the  comments  and  examples  made  by  the  SMEs  when 
providing  contextual  relevance  to  their  answers  during  the  knowledge  elicitation  sessions. 

The  themes  are  listed  in  order  of  frequency  of  occurrence/mentioning. 

1 .  System  Capabilities  (7) 

2.  Personnel  Capabilities  (6) 

3.  Tactical  infonnation  (3) 

4.  Administrative/Logistical  (3) 

5.  Case  Studies  and  Applications  of  Doctrine  (1) 

6.  Strategic  information  (Planning)  (1) 
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Strategic  Information 
Case  Studies 
Administrative  Information 
Tactical  Information 
Personnel  Capabilities 
System  Capabilities 


■  Frequency  Count 


Figure  23.  Table  of  information  best  conveyed  through  Socrates  Window. 

Although  all  of  the  SMEs  agreed  that  Socrates  Window  has  many  benefits  for  use  in 
classroom  environments  (such  as  the  MC3),  and  when  used  primarily  in  a  supporting  or 
supplementary  role,  each  SME  pointed  out  that  such  technology  would  only  be  useful  to  SGIs 
and  the  Soldiers  in  these  courses  if  the  tool  didn’t  require  an  inordinate/exorbitant  amount  of 
effort  to  use.  The  specific  concerns  addressed  by  the  SMEs  were: 

•  Amount  of  time  to  collect  and  integrate  outside  expert  feedback 

•  Amount  of  time  necessary  to  vet  outside  expert  feedback 

o  Vet  for  content 
o  Vet  for  accuracy 
o  Vet  for  situational  applicability 
o  Vet  for  legality 
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Measures  of  Use 


We  utilized  a  COTS  product,  Google  Analytics,  to  monitor  and  quantify  the  use  of 
Socrates  Window  during  our  software  development  and  demonstration  (see  Figures  24-28).  The 
purpose  was  simply  to  explore  its  utility  as  a  measure  of  use.  We  recommend  bundling  a 
capability  like  Google  Analytics  with  Socrates  Window. 
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Figure  24.  Measures  of  use  from  Google  Analytics  -  site  overview. 
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socrates.aptima.com 
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Figure  25.  Measures  of  use  from  Google  Analytics  -  people  visited. 
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Figure  26.  Measures  of  use  from  Google  Analytics  -  traffic  sources  overview. 
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Figure  27.  Measures  of  use  from  Google  Analytics  -  traffic  sources  countries. 
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socrates.aptima.com 
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Figure  28.  Measures  of  use  from  Google  Analytics  -  pages  viewed. 
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PHASE  III  -  ENHANCING  VALUE  THROUGH  PEDAGOGY  FOR  BLENDED 

LEARNING 

The  triad  in  Figure  29  is  a  core  innovation — a  derived  requirement — that  motivates  and 
guides  innovation  in  all  elements  of  the  developing  integrated  capability.  The  most  fundamental 
insight  about  this  triad  is  that  the  intent  for  outside  experts  is  to  reveal  the  values,  broadly 
considered,  of  the  culture  with  which  Soldiers  in  a  particular  branch  must  integrate  to  achieve  the 
JEM  necessary  to  operate  effectively  and  adaptively  in  full-spectrum  operations. 


Outside 

Expert 

A 


Instructor 


Student 


Motivation 
&  Emotion 


,  Meaningful 
\  Experience 


Situated 

Experience 


Figure  29.  Framework  for  science  to  support  development  of  pedagogy  for  blended  learning. 

This  was  a  fundamental  insight  because  one  cannot  expect  to  develop  or  acquire  an 
effective  distance-learning  capability  or  blended-learning  capability  without  first  having  a 
comprehensive  scientific  foundation  for  the  pedagogy  with  which  the  educational  technology 
must  be  consistent.  This  pedagogy  must  be  relevant  to  the  Army.  Relevance  in  this  respect  is 
more  than  about  utilizing  methods  of  adult  education.  Programs  of  education  outside  the  Army 
may  be  able  to  pay  lip  service  to  values-based  requirements  and  the  associated  development  of 
individuals  but  this  is  a  gravely  serious  matter  in  the  Army  as  an  organization,  and  it  has 
existential  significance  to  Soldiers  and  their  families.  We  recommend  that  further  research  be 
conducted  to  identify  and  develop  an  appropriate  pedagogy  for  values-based  education  in  the 
Army.  We  believe  the  central  innovation  of  JEM,  to  link  cultural  insiders  with  experts  from  a 
different  culture,  provides  a  source  of  relevant  case  studies  for  development  and  assessment  of 
values-based  approaches  to  Army  training  and  education. 
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APPENDIX  A:  LIST  OF  ACRONYMS 


ACEP 

Army  Center  for  Enhanced  Performance 

ADA 

Air  Defense  Artillery 

AO 

Area  of  Operations 

BCKS 

Battle  Command  Knowledge  System 

COE 

Contemporary  Operational  Environment 

COIN 

Counterinsurgency 

COL 

Colonel 

COTS 

Commercial  Off  The  Shelf 

DAU 

Defense  Acquisitions  University 

DoD 

Department  of  Defense 

ECOE 

Experts  in  the  Contemporary  Operational  Environment 

FAA 

Functional  Area  Analysis 

FNA 

Functional  Needs  Analysis 

FSA 

Functional  Solution  Analysis 

GEL 

Guided  Experiential  Learning 

IED 

Improvised  Explosive  Device 

JCIDS 

Joint  Capabilities  Integration  and  Development  System 

JEM 

Joint  and  Expeditionary  Mindset 

JFC 

Joint  Functional  Concepts 

JOC 

Joint  Operating  Concepts 

JOpsC 

Joint  Operations  Concepts 

MC3 

Maneuver  Captains  Career  Course 

METT-TC 

Mission,  Enemy,  Terrain,  Troops,  Time,  Civilians 

MOE 

Measures  of  Effectiveness 

MOP 

Measures  of  Process 

MOS 

Military  Occupational  Specialty 

NGO 

Non-Governmental  Organization 

OGA 

Other  Government  Agencies 

OOTW 

Operations  Other  Than  War 

OPFOR 

Opposing  Force 

PIA 

Post  Independent  Analysis 

RSR 

Right  Seat  Ride 

R&D 

Research  and  Development 

SBIR 

Small  Business  Innovation  Research 

SGI 

Small  Group  Instructors 

SME 

Subject  Matter  Expert 

SO 

Stability  Operations 

S&T 

Science  &  Technology 

TDE 

Tactical  Decision  Exercise 

Up 

Tactics,  Techniques,  and  Procedures 

2LT 

Second  Lieutenant 

A-l 


